Cascading Syslog Servers

, , ,

Cascading Syslog Servers

Introduction to the Syslog Protocol

Syslog is a protocol that allows systems to send Event Notification Messages through IP networks to Syslog Servers (also known as Event Message Collectors). There the messages can be sorted, searched, and analyzed to monitor the state of individual devices as well as the overall system.

Syslog messages contain both status information and a Severity Level, which ranges from 0 (zero) to 7. Level 0 messages are emergencies. Level 7 messages signify that the sender is in Debug mode. The meanings of Levels 1 through 6 are application dependent.

2. Multiple Syslog Servers – The Traditional Approach

In some situations you might want to add additional Syslog Servers to your system. Traditionally you would do this by configuring each connected device or server to send messages to the Main Syslog Server and to each Secondary Syslog Server. This configuration is shown in the following image:

This works fine if you just have a few devices. But it quickly becomes impractical as the number of connected devices grows. Imagine configuring 1000+ devices to send Syslog messages to one or more additional servers for a special project, then disconnecting them all later.

This makes the traditional approach impractical for large installations.

3. Multiple Syslog Servers – The Cascading Approach

To avoid the problems of the traditional approach, Nectus implements Cascading Syslog Servers. Instead of connecting each device to each Syslog server, you need only connect them to the primary Syslog server. The primary server can then forward copies of the messages to any secondary servers, as shown in the following image:

This approach makes adding and removing secondary Syslog servers simple. However, forwarding every Syslog message does increase the load on the primary Syslog server. You need to carefully monitor the primary server to avoid overloading it.

Nectus recommends you cascade no more than 10 secondary Syslog servers to avoid overloading the primary server.

3.1 Configuring the Nectus Cascading Syslog Servers Solution

Follow these steps to configure Cascading Syslog Servers:

  1. Click Settings in the Nectus Home Screen.
  2. In the Settings menu that appears, hover the cursor over the General Settings option.
  3. Click the Syslog Settings option that appears. Select the Forwarding IP tab in the Syslog Settings dialog that appears.

  1. Click the Add New IPv4 button to open the Add Forwarding IPv4 dialog.

  1. For each secondary Syslog server add the IPv4 Address of the server, the number of the UDP Port the server is listening on, and a Description of the server.

WHICH US CITY USES THE MOST IPV4 PUBLIC ADDRESSES?

, ,

https://www.nectus5.com/wp/wp-content/uploads/2018/01/Map_USA-01-1500x630.png

Note: Information is based on Nectus IP geo-location service

State City IPv4 Addresses
Ohio Columbus 225326103
California Los Angeles 54776440
Arizona Fort Huachuca 54644594
Texas Houston 42689210
District of Columbia Washington 32721834
New York New York City 31867103
Virginia Ashburn 31828300
Indiana Indianapolis 26421929
Georgia Atlanta 25527566
California Palo Alto 25105708
Washington Redmond 24885468
Michigan Dearborn 23705811
North Carolina Durham 21588969
New Jersey Newark 21491795
California San Diego 21485402
Illinois Chicago 20074587
North Carolina Raleigh 18955414
New Jersey Bedminster 17843408
Texas Richardson 17241943
Texas Dallas 16869204
Massachusetts Cambridge 15868348
California San Jose 15336783
Washington Seattle 15260827
Alabama Montgomery 14906638
California Cupertino 13954110
Washington Bellevue 13800919
Connecticut Fairfield 13507953
California San Francisco 12561267
Pennsylvania Philadelphia 12464449
Virginia Reston 11731922
Florida Lake Mary 10572081
New Jersey Mount Laurel 10087552
Colorado Denver 9869523
Missouri Saint Louis 9426794
California Norwalk 9273764
Virginia Virginia Beach 9107341
Michigan Ann Arbor 8772940
California Mountain View 8474238
Connecticut Middletown 8241397
Texas San Antonio 7877211
Texas Austin 7734993
Arizona Phoenix 7649529
Oregon Portland 7600141
New Jersey Rahway 7312241
Florida Miami 6713810
Ohio Cincinnati 6688810
California Concord 6607183
Virginia Dulles 6470388
Missouri Town and Country 5898488
Massachusetts Boston 5557232
Louisiana Monroe 5300043
Colorado Greenwood Village 5070591
Pennsylvania Pittsburgh 4780729
Missouri Kansas City 4578123
Virginia Herndon 4492530
Michigan Detroit 4336217
Pennsylvania Doylestown 4203957
North Carolina Charlotte 4085710
Tennessee Nashville 3916537
Georgia Duluth 3805720
Nevada Las Vegas 3792683
Illinois Naperville 3716723
Florida Orlando 3665033
California Sacramento 3601243
Utah Salt Lake City 3592200
Alabama Redstone Arsenal 3428226
Minnesota Minneapolis 3412363
Florida Tampa 3400441
New Jersey Morristown 3304100
California Santa Clara 3252933
New York Rochester 3189712
Maryland Baltimore 3079657
Minnesota Saint Paul 3019512
Arizona Kingman 2983075
Massachusetts Springfield 2927039
Wisconsin Milwaukee 2811053
Colorado Fort Collins 2752782
Wisconsin Madison 2732615
California Belmont 2725536
Texas Plano 2671935
Virginia Arlington 2668836
Connecticut Stamford 2609471
Ohio Cleveland 2600011
Kansas Overland Park 2528866
Texas Irving 2512563
Kentucky Richmond 2509411
Texas Fort Worth 2494944
Arkansas Little Rock 2446145
Florida Jacksonville 2423627
Missouri Columbia 2266295
Oregon Beaverton 2224613
New York Buffalo 2210272
California San Ramon 2131203
Ohio Akron 2098568
California Pleasanton 2097585
Maryland Rockville 2072266
California San Mateo 2044008
Nebraska Omaha 2020147
New York Albany 2018827

 

Editing Site Properties in Nectus

, , ,

Edit Site Properties

You can easily edit the properties of any existing Site. In this chapter, you’ll learn how to edit the properties of an existing Site. We’ll also discuss the functions of each editable property.

Opening the Edit Site Properties Dialog Box

It takes just a few clicks to open the Edit Site Properties dialog box for any existing Site. Follow these instructions:

  1. Click Sites in the Sites Panel on the Nectus Home Screen. The “All Sites” list appears.
  2. Open the All Sites list by clicking the plus sign ( + ) to the left of the list.
  3. Navigate to the Site you want to edit and right-click it.

  1. In the menu that appears, select Properties. The Edit Site Properties dialog box appears.

  1. Make your desired edits (details about each editable property follow). Click Ok when done.

Editable Properties

You can edit the following properties from the Edit Site Properties dialog box. Here is a complete explanation of each field:

  • Site Gateways Button: Opens the Site Gateways dialog box and displays all the potential gateway Devices for this Site.
  • Site Level Name: A unique identifier for the Site Level. You can assign any name you wish.
  • Site Specific Hostname Prefix: Setting a hostname prefix here will automatically assign to this Site any unassigned Devices with the same prefix. This feature is currently disabled.
  • Site Color: The color to use for Devices that are assigned to this Site. Click the colored block to select a color, or enter the ASCII color name.
  • Outage Map Icon: A clickable list of icons that can be used to identify this Site. The options are Circle, Triangle, or Star.
  • Outage Map Icon Size: Allows you to adjust the size of the icon that represents this Site. Typically used to make the icons for more important Sites larger.
  • Address: The street address of the Site. Can be used to generate the GPS Latitude and Longitude of the Site.
  • Find GPS from Address Button: When you click this button, Nectus uses Google Maps to find the GPS Latitude and Longitude of the Site.
  • GPS Latitude: The Latitude of the Site. You can enter a value manually, or let Nectus populate this field using the “Find GPS from Address” button.
  • GPS Longitude: The Longitude of the Site. You can enter a value manually, or let Nectus populate this field using the “Find GPS from Address” button.
  • Maintenance Window: Shows the time frame when Site administration is allowed. Click to select from a list of available maintenance windows. These windows are defined at Settings / General Settings / Scheduled Maintenance Settings.

 

Which US city uses the most IPv4 public addresses?

, ,

Note: Information is based on Nectus IP geo-location service

State City IPv4 Addresses
Ohio Columbus 225326103
California Los Angeles 54776440
Arizona Fort Huachuca 54644594
Texas Houston 42689210
District of Columbia Washington 32721834
New York New York City 31867103
Virginia Ashburn 31828300
Indiana Indianapolis 26421929
Georgia Atlanta 25527566
California Palo Alto 25105708
Washington Redmond 24885468
Michigan Dearborn 23705811
North Carolina Durham 21588969
New Jersey Newark 21491795
California San Diego 21485402
Illinois Chicago 20074587
North Carolina Raleigh 18955414
New Jersey Bedminster 17843408
Texas Richardson 17241943
Texas Dallas 16869204
Massachusetts Cambridge 15868348
California San Jose 15336783
Washington Seattle 15260827
Alabama Montgomery 14906638
California Cupertino 13954110
Washington Bellevue 13800919
Connecticut Fairfield 13507953
California San Francisco 12561267
Pennsylvania Philadelphia 12464449
Virginia Reston 11731922
Florida Lake Mary 10572081
New Jersey Mount Laurel 10087552
Colorado Denver 9869523
Missouri Saint Louis 9426794
California Norwalk 9273764
Virginia Virginia Beach 9107341
Michigan Ann Arbor 8772940
California Mountain View 8474238
Connecticut Middletown 8241397
Texas San Antonio 7877211
Texas Austin 7734993
Arizona Phoenix 7649529
Oregon Portland 7600141
New Jersey Rahway 7312241
Florida Miami 6713810
Ohio Cincinnati 6688810
California Concord 6607183
Virginia Dulles 6470388
Missouri Town and Country 5898488
Massachusetts Boston 5557232
Louisiana Monroe 5300043
Colorado Greenwood Village 5070591
Pennsylvania Pittsburgh 4780729
Missouri Kansas City 4578123
Virginia Herndon 4492530
Michigan Detroit 4336217
Pennsylvania Doylestown 4203957
North Carolina Charlotte 4085710
Tennessee Nashville 3916537
Georgia Duluth 3805720
Nevada Las Vegas 3792683
Illinois Naperville 3716723
Florida Orlando 3665033
California Sacramento 3601243
Utah Salt Lake City 3592200
Alabama Redstone Arsenal 3428226
Minnesota Minneapolis 3412363
Florida Tampa 3400441
New Jersey Morristown 3304100
California Santa Clara 3252933
New York Rochester 3189712
Maryland Baltimore 3079657
Minnesota Saint Paul 3019512
Arizona Kingman 2983075
Massachusetts Springfield 2927039
Wisconsin Milwaukee 2811053
Colorado Fort Collins 2752782
Wisconsin Madison 2732615
California Belmont 2725536
Texas Plano 2671935
Virginia Arlington 2668836
Connecticut Stamford 2609471
Ohio Cleveland 2600011
Kansas Overland Park 2528866
Texas Irving 2512563
Kentucky Richmond 2509411
Texas Fort Worth 2494944
Arkansas Little Rock 2446145
Florida Jacksonville 2423627
Missouri Columbia 2266295
Oregon Beaverton 2224613
New York Buffalo 2210272
California San Ramon 2131203
Ohio Akron 2098568
California Pleasanton 2097585
Maryland Rockville 2072266
California San Mateo 2044008
Nebraska Omaha 2020147
New York Albany 2018827

When it comes to Network Discovery we are absolute market leaders..

When it comes to Network Discovery we are absolute market leaders and can determine router model by the sound of its Fan. Just kidding..

But here is how we actually do it.

Let say IP address 10.20.45.1 is alive and we need to determine what type of device this is.

Step 1: Perform SNMP Get operation for sysObjectID.0 (1.3.6.1.2.1.1.2.0)

snmpwalk -v2c -c public 10.20.45.1 sysObjectID.0

This OID stores platform specific string which suppose to be unique for each device type.

For example device responds with: .1.3.6.1.4.1.9.1.924

This string is called Platform Specific OID and contain Vendor code in seventh position.

Each vendor has IANA assigned unique number listed here https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers

In our case code is “9” which means that this is a Cisco device.

The remaining numbers define platform info and that information is collected from MIB files published by each vendor which we all collected and combined into a single repository of 1200 vendors which contains now around 56,000 different platform OID which we all classified by Device Category and Model.

At the end we have very nice device classification tree like this

 

Integrating Cisco Virtual Internet Routing Lab (VIRL) with Nectus

,

In this article we will see some of the features of Nectus that can enhance the Virtual Internet Routing Lab (VIRL) experience.
VIRL is Cisco’s network simulation platform where you can run Cisco OS (IOS, IOS XE, IOS XR, NX-OS, ASA) virtual machines and other third party virtual machines (this includes Linux servers, traffic generators and other networking vendors virtual machines) to build topologies for feature testing and validation before introducing them in production.

We will explain some features of Nectus that can complement VIRL to help you visualize better your network.

This is the VIRL topology that will be used:

The devices were started with some predefined configuration that included interface IP configuration, routing protocols (EIGRP, BGP).
VIRL topology is using shared flat network so that each device will get an IP address from 172.16.1.0/24 network on their GigabitEthernet0/0 interface as their management IP address.

Nectus was installed on a Windows 2016 server that was acting as an OpenVPN client connecting to VIRL server which means that it received an IP address from the range 172.16.1.20 – 172.16.1.39, thus making the Nectus and the VIRL routers to be in the same subnet.

Once Nectus starts discovering the devices from 172.16.1.0/24 subnet (as per discoverable subnets configured on Nectus), it builds a list with them categorizing them based on the vendor, type of the device, model of the device.

Based on the information collected through SNMP, Nectus can build L2 and L3 topologies.

This is the L2 topology:

And this is the L3 topology:

One interesting feature that Nectus can do is to give you a visual result of the path between two points in the network.
This is called L3 Path Discovery (for now only available for IPv4). Source IP, source router and destination IP are the input values:

And the result looks like this:

The interesting part is that it can discover asymmetric paths in the network to give a better understanding about how traffic flows in the network.
Another interesting set of features is that you can get real time graphs with the some of the characteristics of the interfaces (utilization, availability, errors, dropped packets, traffic volume).
This is how you can select any of the graphs. This is for utilization:

And the graph looks like this:

Observe that although on when we selected the link that appear to be between R5 and R4, it is actually between R5 and SW2.
The errors graph shows how many RX and how many TX errors are on interface basis:

You can have a consolidated view of the top most utilized interfaces or the interfaces that have the most errors.
By default, there are few network monitoring dashboards (you can create your own to better accommodate your monitoring needs).
The high level dashboard gives you the top interfaces with regards to various interface statistics:

From here, you will get the the list of interfaces:

Nectus can trigger alerts based on any of these interface characteristics.
For testing purposes, the threshold level for interface utilization at which the alarm is triggered was configured at 1%.
Using ping command (between R4 and R5, therefore through SW2), the interface utilization was around 870Kbps and after changing the bandwidth of the interface to 10Mbps

(adding bandwidth knob under interface configuration), this 870Kbps turned out to be around 8.5% interface utilization which means that the alert should have been triggered.
After some time, the graph is adjusted with the new value:

The alerts log shows these type of alerts:

And in this interface utilization, this is the alert:

Another useful information that can be retrieved directly from Nectus using the interface graphs is the interface availability that can quickly give some hints about service interruption.
The graph is selected from the link menu:

And it should show the state of the interface:

Coming back to the default network monitoring dashboards, the information that an interface that is down is captured by both default dashboards. This is the low level dashboard:

As well by the high level dashboard:

Again, there is an alert sent for such events:

There is a history kept for each outage of the interfaces showing for how long the interface was down:

Going further graphs for interface errors and dropped packets are useful to troubleshoot network performance.

And for dropped packets:

Coming back to alerts, Nectus can monitor the CPU usage and trigger alerts as required.

The Device Info menu contains among other CPU usage graphs.

If the CPU usage goes above the threshold, not only you will see this on the graph, but it will also trigger an alert:

Another interesting feature that can help you quickly find all sort of information about the devices in your topology is the Composite Search feature:

It can find various information and for instance, I would like to find where is this IP configured:

And the result is this:

Lastly, one feature that can improve VIRL usability is that Nectus can show on the topology that a link is down

(after the link was shutdown from CLI or went down for other reasons like err-disable).

Suppose you do this on CLI:

R4(config)#int gi0/2
R4(config-if)#shut
R4(config-if)#
*Dec 29 17:35:29.750: %DUAL-5-NBRCHANGE: EIGRP-IPv6 1: Neighbor FE80::F816:3EFF:FE84:2418 (GigabitEthernet0/2) is down: interface down
*Dec 29 17:35:29.752: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 10.0.128.2 (GigabitEthernet0/2) is down: interface down
*Dec 29 17:35:31.725: %LINK-5-CHANGED: Interface GigabitEthernet0/2, changed state to administratively down
*Dec 29 17:35:32.725: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/2, changed state to down
R4(config-if)#

Then the link will blink on the topology, while VIRL will not show anything with regards to the fact that the interface between the VMs is down:

The up/down status can be enabled from the device settings like this:

Throughout this post, various features of Nectus have shown how Nectus5 can bring value to topologies running in Cisco VIRL.
Of course the same features can be used to know your real/production network better, but it is good to know that you can use Nectus5

to monitor your proof of concept network deployed in VIRL.

 

Selecting fastest Database Engine for Netflow Storage

Well, if you are reading this article you probably know what NetFlow is and how much data it can generate. 100GB of NetFlow records per day is not something totally unusual in NetFlow business.

We tried most of the commercial and open-source existing database engines like Microsoft SQL Server, MongoDB, PostegreSQL, MySQL or even Oracle and were not happy with results.

We tried turning off all the indexing, implemented daily partitions,  decided not to store some less important NetFlow fields  and may be even

upgraded your storage to those fancy M2 SSD and still NetFlow reports took minutes to appear.

If your NetFlow rates are somewhere under 1,000 flows per second you can skip this reading, pick any  DB and it will produce acceptable results.

Problems becomes visible when your flow rates  reaches 2,000 fps and at 10,000 fps just doing INSERT to your tables takes 70% of the time.

10,000 flows per second produces 600,000 database records every minute and only INSERT statement takes around 40 second to process leaving only 20 seconds of

available time  for any reports to be generated. The main problem here is that conventional DB engines are not optimized for storing read-only sequential data such

as time based event logging or NetFlow. Best database engine for Netflow has to be designed with read-only sequential access in mind, no “Delete-Update” functionality is required for NetFlow.

This restriction allows great simplification of DB internal formatting structure  and processing logic. Second DB feature that best suited for NetFlow is reduction of possible Indexes to one.

There is absolutely no value in having indexes for Source/Destination IP addresses or Source/Destination ports as those indexes only benefit single type of NetFlow report

and each index will double your table size on disk. The only Index that is used in all reports is Index by flow time stamp as all NetFlow reports are focused on very specific time frame.

And last but not least DB feature that is required for perfect NetFlow storage is hardware optimization. Allocating each DB thread to a dedicated CPU core  has shown

to increase query processing time by 10x.

When  developing our own NetFlow collector we tried all well-known DB engines with one performing slightly better than others but none of them were able to support the golden standard of 30Kfps.

This was until we met the ClickHouse, open source DB engine developed by Yandex. ClickHouse has a long list of limitations, but those limitations are implemented with

a single purpose to have the fastest logging DB engine available on the market. With a help of ClickHouse Nectus can process 50,000 flow per second in single VM which  is currently

a record among all commercially available NetFlow collectors.

Download your 60-day Trial of the best NetFlow collector.

 

 

 

 

 

Next Generation of Network Discovery and Monitoring Tools.. and NetFlow

Next Generation of Network Discovery and Monitoring Tools

There are many reasons why your network needs to be monitored. It is essential to any network administrator to keep track of the network’s performance and usage in real time and to detect failures, slowness or any other threat that could be affecting the network. Every device that is added to the network, every change in the topology or any failure needs to be immediately detected. That is why we need a network monitoring software to make the best of our network and that is exactly what Nectus is for.

Nectus is a network discovery, monitoring and visualization software and its main role is to discover network topology, generate a visual network diagram and keep it up to date, detecting any failure or unusual behavior that could be affecting the network and alerting the network administrator immediately.

These are the Nectus key features:

  • Automatically discovers connections between devices (via Cisco Discovery Protocol) and stores all this connections in a Database.
  • Network discovery is run everyday and if any new device has been added to the network, the topology is updated automatically so network diagrams will always be up to date, no need to do it yourself so you will be saving your time. You can generate L2 and L3 network topology in just one click, way better than Microsoft Visio!
  • Real time monitoring is overlaid on top of network diagrams. You will be able to see utilization, errors and dropped packets or up/down status directly on your diagrams.
  • Automated configuration backup and configuration changes for routers and switches along with best practice audits so you can find and fix misconfigured devices.
  • Includes free integrated NetFlow collector.
  • Syslog Server: Store unlimited number of Syslog messages.
  • URL Monitoring: Monitor UP/DOWN and latency for any URL.
  • Track configuration differences. Easily find differences in configurations before and after the change.
  • Layer 3 Traffic Path Visualization: See how packets travel from A to B.

If you are looking for a software to keep your network in a best shape, then Nectus is your best choice!

You can download a 60 days fully functional demo at http://nectus5.com/download/ and try all these features yourself.

 

The best network management software just got better.. IPAM

We will be adding IPAM (IP Address Management) module to the Nectus NMS starting from January 2018.

No special licensing will be required and it will be immediately available to all current Nectus users free of charge.

Support for  IPv4 and IPv6 address space management, API interface for third party integration and many more cool features ..

Download Nectus

CircuitDB functionality added to Nectus NMS

, ,

CircuitDB gives ability to track of all the telco circuits (Internet, MPLS, T1 etc), carrier contracts, cabinet/rack/patch panel information.

Support for configurable email alerts on approaching circuit contract renewal dates, integration with real time circuit monitoring.

Never pay for circuits that not being used and  many more cool features.

 

 

How to Manually Start Network Discovery

,

Normally Discovery process runs every night or every weekend, but sometimes it is required to manually initiate Network Discovery at this specific moment.

User has two options:

Option 1 (Discovery starts within 6 min)

Change the value of “Minimum  Interface Between Discoveries” to smallest possible

value of 0.1 hour (6 min) and Network Discovery will start withing next 6 min.

 

 

Option 2 (Immediate Start)

If waiting for 6 minute is not an option then user can clear discovery log  “Logs -> Network Discovery” by deleting all discovery log records

and restart Discovery Service in “Settings -> Service Status”

 

 

When Discovery service is restarted and Discovery log is empty , Discovery process starts immediately.