Multicast Validation Testing of Plant Floor Applications

This testing was for a tier 1 automotive manufacturer that The INS Group had worked for, for many years. Our engineers at that time made up about eighty percent of their Network Architecture Team (NAT), so we were naturally engaged from the onset. The long term initiative of this project was to enable multicasting first in manufacturing and then extend it to the rest of the enterprise. The key impetus for this project were a number of plant floor applications that would benefit significantly by being migrated from broadcast based communication to multicast. 

As was the case with many other projects for this customer, the first step for us would be to determine the technical feasibility of bringing a new technology, multicasting in this case, seamlessly into their network environment. So after our informal evaluation of the multicast capability of the current network architecture, it appeared feasible that this could be done without significant hardware upgrades. 

The formal testing for this project was broken into three phases. Phase I testing was primarily to gain a better understanding of how the plant floor applications were using broadcasting for their inter application communication. Phase II testing would be to validate/test a number of different application scenarios that ran in single and multivendor network LAN architectures. Phase III was similar but for a MAN architecture. To find the ideal multicast solutions for both phase II & III several different multicast methods would be evaluated. 

After the testing and evaluation phases are successfully completed the new validated/approved multicast architectures can be incorporated into the overall enterprise network architecture. After which the deployment part of this project can proceed.

The following outline is a subset of the 156 page multicast test plan made up of technical overviews, 36 test scenario definitions, 330 test cases/instances, configuration/setup and test results.

1. Executive Summary

2. Testing Summary & Conclusions

3. Multicasting Methodology for Single Site and MAN Architectures

4. Phase II & III Network Configuration Overview

5. Cimplicity Network/Application Architecture Configuration Summary

6. An Overview of the Test Plan Methodology 

7. Structure/Layout of each test scenario

8. Test Scenario Definitions

Executive Summary

The plant floor networks that require multicasting are primarily Cabletron switched Ethernet/ATM environments that are flat architectures made up of multiple ELANs with routed subnets into those ELANs. This type of architecture essentially creates one broadcast domain per ELAN. In these type of environments, deploying broadcast based Cimplicity is relatively straightforward. Most of the devices are in the same ELAN (by design), with helper addresses on the LAN routers to go between ELANs. The new Cisco based architecture that has started being deployed over the past several years is IP VLAN based and allows for better traffic segmentation/isolation. This however can be problematic for Cimplicity based applications that are configured for broadcast based communication. Servers are typically placed in one VLAN and the clients in other VLAN’s, which requires IP helper addresses and IP directed broadcast features to be configured on the LAN routers for every client VLAN that needs to see Cimplicity project advertisements. Cimplicity is also capable of advertising projects using multicast communications. Several manufacturing sites have successfully deployed one off tactical multicast based Cimplicity solutions in Cisco only architectures. Other sites have tried and were not successful for reasons not fully understood at this time. The problem becomes even more complicated when trying to deploy Cimplicity in a mixed Cisco Cabletron environment. Many assembly and powertrain plants have mixed Controls, legacy Cisco and legacy Cabletron networks that will need to share Cimplicity data. These environments are considerably more complex  and require further evaluation. The following test document will evaluate the known operational scenarios at these plants that will require multicasting. With the primary objective being to develop and deploy a strategic multicast architecture across all manufacturing. Following is an overview of the methodology, configuration and several test scenario examples:

Testing Overview, Summary and Conclusions

Test scenarios 1 through 12 were done in phase I, phase II consisted of nine new scenarios. Phase I testing primarily focused on broadcast based Cimplicity and started preliminary multicast testing. A comprehensive multicast solution was not found in phase I of this project. Phase II would only concentrate on multicast based Cimplicity, continuing were phase I left off. Phase II also required this new multicast based architecture to operate seamlessly in both Cisco and hybrid Cisco/Cabletron environments. A multicast based Cimplicity solution is essential if the new Cimplicity blockpoint is to be deployed at hybrid Cisco/Cabletron brownfield sites. It is also imperative that multicast based Cimplicity work in PNTA deployed plants. The plants will need Cimplicity to be able to communicate between a Cisco/Cabletron environment during migration from Cabletron to PNTA. Broadcast based solutions would be impractical due to the extreme complexity of the IP helper addressing that would be necessary in the predominantly layer three architecture of PNTA. A multicast based solution will also eliminate the inherent security risks (denial of service attacks) created when ip helper addresses are used to forward broadcast packets.

Phase II Cimplicity multicast testing went very well and a solid understanding of how to configure the Cimplicity application and network to operate through multicasting were established through this project. The testing in this project was made up of nine test scenarios, the details of which are described throughout the test documentation. Each of the test scenarios describes briefly what is to be tested and then goes on to define the equipment, setup/configuration, test procedure and results. If the test procedure is successful which means that the network and Cimplicity application for this particular scenario work, the testing then moves to a failover test mode. Each test scenario has twenty six failover tests that fail end device connections, trunk links, router links both soft and hard, ATM links and power failures. The only test scenario that had unresolved multicast failover issues was scenario nine where the Cabletron network was setup in 802.1Q mode. Failing LAN router links, ATM links and power failing the routers and ATM switches resulted in loss of multicast routes and the Cimplicity application loosing project advertisement communication between servers and clients. It is important to note that the devices in question are only on the Cabletron network. This is not a major concern today since there are only two sites in GM where Cabletron is setup in 802.1Q mode; SPO World Headquarters and Saginaw Malleable Iron. This test scenario was investigated primarily because converting to 802.1Q has been considered at Cabletron sites to enable connection of industrial Ethernet switches (which also run 802.1Q). There are currently no plans that the Staging Center is aware of to implement 802.1Q at any GM facilities and a note about the ramifications of test scenario nine will be added to the Staging Center networking best practices document. Test scenarios 1-8 worked very well with no multicast failures. After each failover test the presence of project advertisements on each client and server was verified. The multicast routes on the LAN routers were also checked to verify that all necessary multicast routes required to allow visibility of project advertisements, existed. 

There were failover tests that took up to ten minutes to recover and several times a Cimplicity server needed to be rebooted due to loss of TCP/IP connection oriented communication. These failover issues where not multicast related, as mentioned above the multicast based project advertisements always recovered, typically within 15 seconds. The failover problems that were observed were with standard TCP/IP unicast communications. The failover tests that typically caused these issues were failing the ATM switches, ATM links and occasionally the Cabletron LAN router links. The root of this problem is due to an ATM connection timeout on the up stream Ethernet switch that can take up to ten minutes to clear. At this time without further testing at a site, the assumption is that the same tests done at any Cabletron site would have the same results. The Cabletron test environment in the lab was setup to reflect a typical GM manufacturing environment. The core was made up of two ATM switches and two 6000 switches, the distribution layer had two 2200 switches setup as separate IC’s with an access layer 2200 switch attached (see Cimplicity Multicast Testing Architecture Diagram 1 for more detail).

A summary of the router and switch configurations used for this testing that enables multicast communication on mixed Cisco/Cabletron networks can be found in sections 3, 5 and 6

Multicasting Methodologyfor Single Site and MAN Architectures

Multicasting is also needed in MAN environments so a phase III was required. Phase III testing will focus on Multicast source discovery protocol (MSDP), anycast RP (rendezvous point) and protocol independent multicasting (PIM) as the core multicast technologies to evaluate. MSDP is used to discover multicast sources in other domains. There are many production sites that are part of a MAN, each one will need to be a separate domain as defined in PFCN. All legacy Cisco and Cabletron installed sites will be moving to PFCNA networks in the near future. This testing project must develop an architecture that will seamlessly mesh together with the PFCNA. MSDP is also used extensively within the PFCNA plant and campus architectures. The MAN multicast architecture will adhere to the VLAN and IP addressing schemes set forth in the PFCNA template. 

The multicast configurations developed during this testing are more complex than the previous phase II project which only used Anycast RP. The inherent complexity of enabling a new protocol across MAN’s, adhering to controls architecture and being able to control the multicast groups allowed in and out of a site was a challenge. The MAN routers were setup as a fully peered MSDP mesh group controlling the MAN domain. They are peered through their Loopback0 interfaces using the “ip msdp peer ‘ip-address’ connect-source Loopback0” command for each router to be peered. The command “ip msdp mesh-group MAN ‘ip-address’” was used for every router in the mesh group. The command “ip msdp originator-id Loopback0” allows the router to use an interface address other than the RP address (the default) when originating Source-Active messages. This is necessary since all routers in this mesh group make up one domain using the same RP on each. The multicast groups allowed on the MAN are controlled by access lists applied to the MAN RP. The commands used for this were “ip pim rp-address 120.XX.XX.XX ‘ACL #’” to select the RP and “access-list # permit ‘multicast-address’” to permit the multicast addresses used on this RP. The multicast groups allowed on the LAN are controlled in the same manner but with an additional “ip pim rp-address 120.XX.XX.XX ‘ACL #’” and “access-list # permit ‘multicast-address’” commands. To enable a multicast group to be local only it would have a permit entry in the local only access-list. To enable a MAN multicast group to be used locally it would have a permit entry in the LAN routers MAN access-list, it would also have to have identical entries in the MAN routers ACL. Effectively what is being done here is to select the RP the routers will use for different multicast groups (see the multicast configuration and troubleshooting sections for more details). The access lists setup for testing allowed only single multicast entries but could have just as easily been ranges of addresses. 

The sections that follow will show the router configurations used in the testing of MSDP based multicasting. Troubleshooting show commands and their use will also be reviewed. 

Phase II & III Network Configuration Overview

There were several key configuration pieces discovered during testing that enabled multicast based Cimplicity to work in the hybrid Cisco/Cabletron networks tested. When a Cabletron network is running in Securefast mode, Securefast acts as a multicast querier. This is a function in a classic network architecture that would normally be performed by a router. What was happening is that Securefast doesn’t see the router as part of a multicast group because the router doesn’t respond to Securefast multicast queries (normal router behavior). Securefast is also not configurable to allow it to for example, run “PIM or MSDP” which could enable compatibility with the routers. The first direction we went in was to turn this function off on the Cabletron switches. As it turns out this is an embedded function that is key to the operation of Securefast and can not be turned off. We then turned our attention to the routers to try and find a way around this problem. What was needed was a way to make the router look like a client and join the Cimplicity multicast group. We found an interface level command “ip igmp join-group ‘multicast-address’” which does just that. Essentially this command forces a router interface to join the multicast group and respond to multicast queries with multicast reports. Securefast now sees the multicast reports, adds this router port to it’s multicast table and starts forwarding multicast traffic for this group to the router. This command was not needed on the Cisco network or on the Cabletron network setup in traditional 802.1D or 802.1Q modes. Another key component in this project was the use of PIM (protocol independent multicast) with anycast rendezvous points (RP’s). Anycast rendezvous points require an interface to be setup as the rendezvous point, normally this is a loopback interface which is what was used. A second loopback interface was also setup on the backup router for redundancy. All of the testing was done with the RP’s on the Cabletron LAN routers.

The phase III multicast testing objective was to find a solution that could tie multiple sites together. These sites could be running legacy Cisco, Cabletron or the new PFCNA (plant floor controls network architecture). This was simulated in the lab by tying together pairs of routers which are interconnected to make up the MAN environment. The lab test environment was setup to simulate three sites, one of each architecture. This more complex environment required a solution that could be extended to multiple sites, allowing the site to have local only multicast traffic and also join remote multicast groups from other sites. Access control lists were used to control which sites could register with multicast groups from other sites through the MAN routers.

The Anycast RP solution developed in phase II is not well suited for this more complex environment. Anycast RP is still used but in conjunction with multicast source discovery protocol (MSDP). MSDP is used to join multiple PIM sparse-mode (SM) domains through source advertisements (SA’s). If a PIM-SM domain has sources or receivers interested in this multicast group, it will register this source and pass it down it’s multicast tree. Multiple domains are tied together through an MSDP peering relationship with other PIM-SM domains. The MAN routers in the lab setup are fully meshed and peered together, each with it’s own RP (same address/domain). These MAN routers effectively act as the global multicast traffic providers. Access control lists tied to the MAN RP can be used to allow only a certain group or a range of multicast groups. The local sites tie into the global groups they need through access lists tied/matched to the global RP; the global RP becomes active locally for the group/range that matched the ACL . Local only multicast groups are joined through access lists tied/matched to the local RP. This will be discussed further in the detailed configuration section that follows.

Cimplicity Network/Application Architecture Configuration Summary

The multicast based Cimplicity testing was done in a hybrid Cisco/Cabletron network joined at layer three. There were nine network configurations/scenarios considered, made up of different hardware platforms and network protocols. The hardware on the Cabletron side with the exception of the LAN routers was fixed throughout all nine scenarios. The Cabletron hardware configuration is typical of most Cabletron sites. It is made up of a core primary and secondary ATM switch, core 6000 server switches, 2200 switches at the distribution and access/edge layers. Two different router platforms (Cisco 4500 and 7301 routers) were used as LAN routers on the Cabletron network. On the Cisco network at the core were 5500’s running Cat OS, a 6500 running IOS on the primary and a 6500 running Cat OS on the secondary. The distribution/access layers were fixed in all scenarios with 3550’s at the distribution and a 2950 and 3750 at two different access layer TC’s. 

The network protocols used for inter-switch communication on the Cabletron network were Cabletron Securefast, 802.1D and 802.1Q. Since there are no 802.1Q Cabletron assembly sites only one test scenario was executed with this protocol setup on the Cabletron network. The Cisco network used 802.1Q and Cisco interswitch link (ISL) for VLAN trunking in all scenarios. ISL is a Cisco proprietary protocol that was included in this testing because there are sites running 5500 series switches that are trunking with ISL. The Cabletron networks are made up of ELAN’s with multiple routed IP subnets. The Cisco network was made up of multiple VLAN’s and a management VLAN. 

The Cimplicity application was made up of four clients and four servers. Two of the servers were running Cimplicity blockpoint version 4.10 and two were running the new blockpoint version 6.11a. A DNS server was setup with global “CIMPMULTIP” name to resolve the multicast address being used for Cimplicity project advertisements. The DNS server eliminates having to touch each client to configure the Cimplicity multicast group address. The Cimplicity clients will automatically try to resolve this name through a DNS server if it is not locally defined. One server on the Cabletron and Cisco networks were also setup with teamed NIC’s.

The following network/applications architecture diagram was used for all test scenarios in phase II Cimplicity multicast testing. The only changes necessary to make the drawing accurate for different scenarios is with the core Cisco switch/routers (6500/MSFC’s or 5500/RSM’s) and the LAN routers (4500’s or 7301’s) on the Cabletron side. 

An Overview of the Multicast Test Plan and Test Scenario Definitions

The MAN will be made up of a legacy Cisco network, legacy Cabletron SecureFast network and a PFCNA network. The MAN routers that will be interconnecting the three networks are two Cisco 7301’s, two Cisco 4500’s and two Cisco 3550’s. Cimplicity clients and servers will be connected to the Cabletron, Legacy Cisco and full layer three PFCNA network. There will be two servers on the PFCNA network and one server on each of the other networks. This will effectively setup multicast sources in each campus which is a more complex multicast based Cimplicity environment than would typically be found. Fail over testing will be performed on all router uplinks and the operation of the Cimplicity application verified, the routers multicast route tables and source caches will also be checked/verified after each failure. 

The previous multicast test scenarios were based on an anycast RP solution. The following multicast test environment will use anycast RP (rendezvous point) and MSDP (multicast source discovery protocol). This solution is aligned with the latest PFCNA architecture and will seamlessly integrate with it. MSDP gives additional redundancy, control and security features not available with anycast RP. The MAN routers in this test will have multiple RP’s in the same domain (RP address) and will be fully peered and meshed. ACL’s will be used on the MAN and LAN routers to permit/direct multicast traffic within the MAN/LAN. 

Servers will be setup for multicast based communications and clients will need to see project advertisements from servers on each network and the servers on each network also need to see each others project advertisements. This will verify/validate that the application is working properly from a network connectivity aspect. One server on the Cisco PFCNA network will also be configured with dual teamed NIC’s.

An IXIA Chariot test utility will be used to generate multicast traffic and load the various networks. It will also be used to generate additional multicast groups that will be setup up in different source receiver combinations. Ideally breakpoint/failure thresholds can be identified within these three networks.

Network General Infinistream multi-port sniffers will be used to trouble-shoot/resolve network configuration issues.

Test Scenario Definitions:

1. Verify that a Cimplicity client/server project works in a pure Cisco Staging Center same subnet/VLAN environment with broadcast enabled.

2. Verify that a Cimplicity client/server project works in a pure Cisco Staging Center multiple subnet/VLAN environment with broadcast enabled.

3. Verify that a Cimplicity client/server project works in a pure Cisco Staging Center same subnet/VLAN environment with multicast enabled.

4. Verify that a Cimplicity client/server project works in a pure Cisco Staging Center multiple subnet/VLAN environment with multicast enabled.

5. Run Cimplicity with a configured non-default multicast address on the client and the server with DNS multicast address to host name resolution in a Staging Center Cisco environment different VLAN’s.

6. Verify that a Cimplicity client/server project works in a pure Cabletron Securefast same subnet/VLAN environment with broadcast enabled.

7. Verify that a Cimplicity client/server project works in a pure Cabletron Securefast multiple subnet/VLAN environment with broadcast enabled.

8. Verify that a Cimplicity client/server project works in a pure Cabletron Securefast same subnet/VLAN environment with multicast enabled.

9. Run Cimplicity with a configured multicast address (non-default/multiple project) on the client and the server with DNS multicast address to host name resolution in a Cabletron Securefast environment same VLAN.

10. Helper addresses are setup in the server VLAN’s/subnets that point to client subnets on both networks (Cisco & Cabletron).

11. Setup severs (broadcast based comm.) and clients on both a Cisco and Cabletron network where the clients will need to see project advertisements from severs on both networks and the servers will also need to see each others projects.

12. Setup severs (broadcast based comm.) and clients on both a Cisco and Cabletron network where the clients will need to see project advertisements from severs on both networks and the servers will also need to see each others projects. Helper addresses from the Cisco network will only be sent to the server subnet on the Cabletron network. This scenario is being done to prove that forwarding broadcasts to more than one interface in a flat single broadcast domain, as was done in scenario 10a, is the cause of the broadcast loop created in scenario 10a.

13. Same as 10b except an additional 4500 router is added to the Cabletron network and HSRP (hot standby routing protocol) is turned on between the two 4500 routers. This test is being done to determine if the router ports in standby will still pass broadcast traffic, which would create a broadcast loop.

14. Same as 10c except an additional RSM is added to the Cisco network and HSRP (hot standby routing protocol) is turned on between the two RSM’s. This test is being done to determine if the RSM VLAN interfaces when in standby will still pass broadcast traffic, which would create a broadcast loop.

15. If a loop is created in 10d, the test will be re-run with IP-DirectedBroadcast and ip-helper-addresses disabled on the backup RSM, which should stop the broadcast loop.

16.  If a loop is created in 10d, the test will be re-run with HSRP off and the interface disabled on the backup RSM, which should also stop the broadcast loop.

17. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. Test scenarios 3, 4 and 7 shows that multicast based Cimplicity will work in a Cisco or Cabletron environment. Verify that Cimplicity will work in a mixed Cisco/Cabletron network were the Cabletron network is running in Securefast mode.

18. Verify that Cimplicity will work in a mixed Cisco/Cabletron network were the Cabletron network is running 1st& 2nd generation switches in an 802.1Q mode.

19. Benchmark Multicast rate on server, what is the interval of Cimplicity project advertisements

20. Benchmark from the server the interval for IGMP member reports, steady state

21. Benchmark from the client the interval for IGMP member reports, steady state

22. Benchmark IGMP queries after link state failure.

23. Benchmark from the client how long it takes for IGMP member queries to be answered with IGMP membership reports after a link state failure; flapping state (make sure they come back). This test is being done to determine how long it should take for the client app to recover from a loss of network connectivity.

24. This testing is being done to find the proper configurations required to get multicast based Cimplicity to work across Cisco and Cabletron networks. Cimplicity servers (Multicast based comm.) and clients will be setup on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. For this test to pass Cimplicity project advertisements must be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode. When this is accomplished the testing will move on to the nine primary test scenarios.

25. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode.

26. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode.

27. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. All servers will be configured with dual teamed NIC’s where each servers individual NIC’s will be connected to a different core switch. Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode.

28. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode.

29.  Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode.

30.  Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in 802.1D mode.

31. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in 802.1D mode.

32.  Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in 802.1D mode.

33.  Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in 802.1D mode.

34.  Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in 802.1Q mode.

35. Multicast Testing with MSDP in a Campus to Campus MAN Environment.

36. Setup severs (Multicast based comm.) and clients on both networks where the clients will need to see project advertisements from servers on each network and where the servers on each network also need to see each others project advertisements. A server on the Cabletron and Cisco network will be configured with dual teamed NIC’s Verify that Cimplicity will work, that project advertisements can be seen by all Cimplicity servers and clients on both the Cisco and Cabletron network were the Cabletron network is running in Securefast mode.