The following test procedures are to evaluate the Meraki Wireless cloud WLAN Controller (MCC) product. The primary goal of this evaluation is to determine if the Meraki system has the functionality required to operate in a Customer office environment. All testing will be performed in a lab that will simulate a Customer Product Development office environment. It should be noted that this testing was not an in depth evaluation and stress test of each feature. The goal is to determine if the product functions as advertised and whether it would easily fit into Customer's network architecture.
The scope is to test the functionality and operation of the Meraki MCC. The goal of this testing is to verify that certain functionality/features called out in the Meraki Cloud Controller Product Manual (MCC-PM) operate as described. The testing done was functional testing, not interoperability/integration testing, not in depth protocol analysis of each function or stress testing of those functions.
Meraki Basic Operation
· AP Configuration changes made on the MCC;
· MCC interface responsiveness
· Verify AP mapping capabilities: PASS
· Verify/Clarify Meraki optimization algorhythms/routines: PASS
· Verify Meraki network operation when MCC is down or unreachable: FAIL
· Verify Meraki mesh networking capabilities: PASS
· Over the Air Upgrades N/A
· Verify VPN's into a Customer environment work properly: FAIL
· Test Dot1q trunking capability
· User Access Control
· Identity Policy Manager: PASS
· Traffic Shaper: FAIL
· Guest management
· Rogue AP Detection: PASS
· Verify Misc. Additional Wireless Features
§ Radio Controls
§ Channel Planning Report
· Test throughput and response time: PASS
· Test speed and duplex from 10/half through 100/full: PASS
· ICMP/Ping testing: PASS
· Get IP addresses through the Staging Center DHCP server: PASS
· Verify that a client can authenticate to through the Staging Center Active Directory server: PASS
· Verify time can be synchronized through SNTP: N/A
· Verify that the internal Cisco AP’s SNMP configuration is recognized by the Staging Center Cisco LMS: N/A
· Verify that the LAP and AP can be managed through web browser, Telnet, console/serial, SSH: PASS, FAIL, FAIL, PASS
· Verify network security through Meraki hosted authentication server: N/A
· Verify network security externally hosted
· Ethernet port failure to the LAP test to verify that the LAP AP powers down
· Power testing
· Wireless 802.11a channel testing: PASS
· Wireless 802.11g channel testing: PASS
· Wireless 802.11b/g channel testing: PASS
· Wireless 802.11n channel-bonding: FAIL
· Mobility Grouping: N/A
· Layer 3 Roaming: Ran out of time to test this
1. In our testing/evaluation of the Meraki RF system several issues have been uncovered that would conflict with Customer's current strategic layer three network architecture. The Meraki AP's although they do communicate to a controller are essentially autonomous access points from a user/data perspective. The AP's control/statistics information is communicated to the Meraki Cloud Controller (MCC) through a secure tunnel. User data packets are tagged with the appropriate VLAN # and put on an 802.1q trunk between the AP and an access layer switch. In Customer's current RF architecture the user and control data would travel through a secure LWAPP tunnel to a controller within the Customer LAN. When the user data gets to the controller it is assembled into an 802.1Q frame, tagged with the appropriate VLAN and put on a trunk linked to a core on the LAN. The problem is that in order for the user data from the Meraki AP to get to the core where the RF user VLAN's are defined, it would need to be trunked from the access layer where it is connected up to the core/dist layer. This is the easiest way to solve the problem but would essentially mean going back to the architecture that Customer started to migrate away from four years ago. The current strategic architecture at Customer is a layer three architecture that eliminates general purpose trunked VLANs to the access layer and the spanning tree issues that come with it. To maintain the layer three architecture and deploy an autonomous RF solution the core based RF user VLANs would need to be defined/created at the access layer. This method would also be a significant change from the current architecture and require extensive rework of existing sites already migrated to the layer three architecture.
2. Another issue, due to the AP's being autonomous is that the AP's will need to be added to the RADIUS server. In Customer's current controller based architecture only the controllers need to be added to the RADIUS server. This would primarily impact QIP before/during deployment and ongoing maintenance. This also complicates network design and deployment.
3. The mapping features on the MCC utilize Google Maps that are over 10 years old.
4. When AP's lose contact with the MCC and are then powered down or reset they appear to go offline and clients can't associate/connect.
5. How secure (physically and logically) is the Meraki cloud? This question is far reaching and beyond the scope of this document but should be evaluated before a pilot on a real Customer network.
6. What are Meraki's change control procedures? Would Customer be effected by new code versions that may need to be applied? There where changes made to the MCC during our testing to help allow communications through Customer's proxy servers to the internet from within Customer. Would Customer have it's own controllers and be able to dictate a code blockpoint scheme for Meraki to follow?
7. What is Meraki's support model? What time-zones does Meraki have support staff? Our experience during this evaluation project was that Meraki support was slow to respond to issues, sometimes taking several days.
8. Are there redundant controllers in the Meraki cloud? How could Customer verify this? Does Meraki have standard failover test procedures? When new code versions are applied, does Meraki do regression testing?
9. After the change was made to the MCC to enable the AP's to get through the Customer proxy we were no longer able to get to "http://my.meraki.com/".
10. The previous issue and also not being able to get to the AP from a browser session essentially makes the AP unusable. A console port would solve this problem.
11. Would Customer be able to have a dedicated pair of MCC's in different regions?
12. Many MCC features are limited i.e. the firewall feature only allows destination addresses.
13. Documentation is limited and lacks detail.
14. In one test scenario an AP was disconnected from the network and it took several hours for the AP to show as missing on the MCC.
15. RADIUS testing issues: Had started testing RADIUS and after getting things setup properly everything appeared to be working properly. I was able to get an IP address, checked the RADIUS event viewer and it showed the test laptop had successfully authenticated with RADIUS and active directory. I started running Chariot traffic between the wireless laptop and a wired client; the throughput was running at about 60 Mbits. When I started testing the next day I wasn't able to get authenticated. I tried a number of times with no success. Worked with Meraki support for several hours and was able to get authenticated again after powering off (battery and plug) the test laptop. Then tried to authenticate again and failed. Through all this the Meraki support person didn't want to cycle power to reset the Meraki AP. The behavior before the laptop power was cycled indicated that the problem was with the AP. So we decided to cycle power on the Meraki AP. Was able to authenticate numerous times throughout the rest of the RADIUS security testing. Didn't have time to try and re-create this issue, so at this time I'm not really sure what the root cause of the problem was.
Following is a list of testing tools and their uses:
Following is the basic format of each test scenario: