Category: Blog

The Single Point of Failure

The Single Point of Failure The Paradox of Orion without High Availability

The SolarWinds Orion platform is used across every industry to monitor systems of all shapes and sizes. Those familiar with Orion have probably seen it used to monitor network devices for performance and up/down status, servers and applications for similar metrics, and a wide range of other functions such as flow monitoring, configuration backup, and change detection. Of all the capabilities of Orion, the most under-the-radar feature still used in almost every environment in some form or fashion is the ability to monitor points of failure within the network.

This takes many forms, including the Groups and Dependency functionality and the ability to monitor High Availability (HA) status for things like Cisco ASAs or F5 Load Balancers. Regardless of how it is implemented or alerted on, the common theme is that organizations are keen to track the status of critical points of failure within their environment. Despite this emphasis on monitoring points of failure within the network, there is an alarming trend among many organizations: relying on a single point of failure for their monitoring system.

In the case of Orion, this is frequently a two-fold problem.

1. Reliance on a single application server

The first problem is the reliance on a single Application Server. While some organizations have the ability to VMotion virtual servers from one host to another in the event of failure, too many are left in a situation where hardware failure would result in the loss of monitoring across the board or, at the very least, the loss of monitoring provided by an additional polling engine.

SolarWinds offers a High Availability solution for primary and additional polling engines that is significantly cheaper than standing up a complete secondary Orion environment that will ensure minimal downtime of the monitoring system in the event of hardware failure. Unfortunately, despite the availability and value of this product, the most common answer when asking Orion Admins about Orion High Availability is, “What’s that?”

2. Neglect of the SQL Database Server

The second major issue is less visible but possibly even more important: the SQL Database Server. SQL has a range of high availability solutions (such as Always On) and backup options that can mitigate or prevent downtime in the event of hardware failure, but all-too-often the Orion SQL Server is set up during the initial deployment and then forgotten about until someone notices poor performance in the Orion application. Even then, the focus tends to be on the health of the database itself, and little mind is paid to the backup and high availability options despite the fact that a failure of the SQL Database Server will cripple the entire monitoring system.

Mitigate the single points of failure

It is absolutely critical that Orion administrators take care to mitigate the single points of failure inherent in their monitoring system. Failure to do so could result in the loss of the very platform they rely on to monitor these same issues in other systems throughout their environment.

Invest in high availability solutions for your monitoring system so a catastrophic event doesn’t kill the tool you rely on to alert you to the incident.

Observability and Monitoring—Siblings, not Rivals

Observability and Monitoring Siblings, not Rivals

The odds are increasing that if you work in IT, you, or someone close to you, has been talking about “observability” recently. And, simultaneously, the conversation likely includes some comparison or question about the difference between observability and monitoring.

So, let’s try to clear up some of this confusion. 

Over the last several years, the charge into observability has spawned largely from developer and DevOps communities that operate primarily in cloud-centric or cloud-native environments. It has been a conversation born of necessity as developers and Site Reliability Engineers (SREs) have sought to create and improve the tools needed to ensure the delivery of robust and secure cloud applications. Developers and SREs from some of the largest, most recognizable cloud software companies have contributed to open source tools and, in some cases, even launched their own software companies around those tools. Martin Mao of Chronosphere and Charity Majors, along with her team, at Honeycomb, spring to mind.

The confusion comes from the overlapping nature of legacy application delivery compared to modern cloud applications and containerized solutions. The historical best practices in the legacy application world leveraged monitoring software and techniques for both the infrastructure used to deliver applications and to monitor the health and availability of the applications themselves.

Another legacy technique that deserves mention here is classical application debugging techniques vs. cloud application debugging techniques. What used to be done via physical serial-based debugging with symbols, etc., for operating systems, or via a small number of application log files for debugging legacy applications, has shifted to cloud-scale complexity featuring distributed application delivery. This has triggered a wave of more powerful solutions designed to ingest, process, and analyze massive volumes of session logs and trace data.

When thrust into the context of most enterprises today, the result is our modern hybrid cloud world and the underlying blend of old and new tools and techniques used to keep everything running smoothly.

So, observability is the language used by those who are predominantly supporting cloud applications, while monitoring persists as the familiar language to those supporting on-premise, legacy, and hybrid environments.

And what most are discovering is that a mature organization uses monitoring and observability, combined with effective incident management, to glue it all together.

So, the question should be, “how do we bring effective monitoring and observability tools and techniques together in our organization?” and not merely, “what is the difference between monitoring and observability?”.

The best solutions combine best-of-breed tools that deliver end-to-end visibility, automated correlation, and actionable insights that keep things running smoothly and support meaningful data analytics. The data from monitoring and observability tools should allow for effective forecasting, planning, and budgeting, going beyond the basic aspects of security, availability, and performance.

At Loop1, that is precisely what we do. With our monitoring maturity model, L1M3*, we combine people, tools, and processes to deliver better business outcomes.


*L1M3 Loop1 Monitoring Maturity Model (LIME)

Watch the full recorded video

Bill Fitzpatrick, Loop1 Chairman and CEO

Bill Fitzpatrick
Chairman and CEO | Loop1


An accomplished engineer with a gift for translating technical concepts into plain English and a sharp business sense, Bill Fitzpatrick is building on the success of Loop1 Systems to execute an ambitious vision for the future. Bill co-founded Loop1 Systems, a SolarWinds Authorized Partner, in 2009. He played an integral role in building the nearly 10-year-old business into what is today, a bustling, Austin-based company counting more than 200 of the Fortune 500 as clients. In the summer of 2017, Bill assumed full ownership of Loop1 Systems and has since laser-focused on one simple goal: to bring the truth to light for each Loop1 client.

We Aren’t Nuts… We Guarantee IT

“I guarantee it.”

Even today, years removed from when the ads were first aired, when I hear that phrase, I can still picture the owner and CEO of Men’s Wearhouse talking about how you would “love the way you look” if you bought a suit from him.

Today, it’s unusual to hear those kinds of promises or guarantees, which is a shame.  So at Loop1, we are reviving this notion.

We guarantee you will get more from your SolarWinds tools and the teams that use them when you engage with Loop1.

George Zimmer | Men's Wearhouse
Certifed Secure Orion by Loop1 Badge

We are launching our “Certified Secure Orion” offer for SolarWinds Orion deployments to kick off this new guarantee.

Our SolarWinds Certified Professional (SCP) engineers will audit your Orion deployment against documented best practices and certify your deployment secure. During the audit, we will identify, document, and remediate minor configuration issues, assuming your environment’s necessary change control permissions are set to allow.

Over the last six months, Loop1 has helped hundreds of Orion administrators remediate their deployments. On many occasions, we observed a need to better communicate to leadership that the environment was fully remediated and ready to be trusted.

Now, you might be asking,

Are they nuts?! How can they possibly guarantee my Orion instance is secure?!

So, I should clarify, our certification is a “point in time” audit and certification exercise.  We can validate that the deployment has been correctly configured and secured, but we can’t promise it will stay that way.

So our certification is not a warranty against future security issues.  Unfortunately, no one can guarantee that, not even our experts! Indeed, there may be times where the work required to remediate a deployment appropriately will be beyond the scope of what the audit can provide, so when that happens, we will let you know.

So, we’ve built the process, vetted it with our attorneys, and we are ready to go.

If you would like to be able to provide your boss with a “Certified Secure Orion” letter from Loop1, give us a call, we’ll help you out.

Certified Secure Orion
Bill Fitzpatrick, Loop1 Chairman and CEO

Bill Fitzpatrick
Chairman and CEO | Loop1


An accomplished engineer with a gift for translating technical concepts into plain English and a sharp business sense, Bill Fitzpatrick is building on the success of Loop1 Systems to execute an ambitious vision for the future. Bill co-founded Loop1 Systems, a SolarWinds Authorized Partner, in 2009. He played an integral role in building the nearly 10-year-old business into what is today, a bustling, Austin-based company counting more than 200 of the Fortune 500 as clients. In the summer of 2017, Bill assumed full ownership of Loop1 Systems and has since laser-focused on one simple goal: to bring the truth to light for each Loop1 client.

Raising a Hand for Mental Health with MIEND


The past year has brought challenges that can be overwhelming for some of us. Daily routines are interrupted, and suddenly, either you or someone you love is facing a mental illness.

As loopsters, we support the families and caretakers of those who face challenges with their mental health.

We are proud of the team in the UK for supporting each other every morning with a quick early meeting to check on everyone since the pandemic began. WE are better than ME.

Derik P. from the US once found a new healthy hobby while working in Germany and arriving when it was dark and leaving work when it was dark early at night. He used the weekends to go outdoors riding his bike. We are purposeful.

May is Mental Health Awareness Month, and the Loop1 CARES committee supports with their “Raise a Hand for mental health” campaign.

Don’t forget to SHARE:





Top 5 things to do as a new SolarWinds Administrator

Top 5 things to do as a new SolarWinds administrator

In this latest blog post, from Loop1 programming guru, and training instructor, Steven Klassen, learn the top 5 things you may want to consider as a new SolarWinds administrator.

So you have a new SolarWinds install and you’ve been handed the keys and the responsibility of monitoring all the things. You know there are probably some settings that should be given attention sooner than later, but which ones?

1. Set Your SMTP Information

There are a lot of things that SolarWinds software will do out-of-the-box, but one of the most important is letting you know when things go wrong. To that end, you’ll need to make sure it knows how to get that information to you.

Under Settings > All Settings > SMTP Settings you can tell the system where your SMTP server is, if it requires authentication, and from whom the emails should appear to be coming from. A common choice for this is where is your company’s domain.

Add SMTP Server

2. Customize Your Data Retention

How much data about your system monitoring do you want to keep? I know the answer is probably “well, all of it…” but realistically you probably care about information that’s recent more than you care about what happened 3 months ago.

By default, SolarWinds stores 7 days of detailed data. That means every time your device is pinged SolarWinds stores a pass/fail in the form of a 0 (it didn’t respond) and 100 (it responded). After 7 days those get averaged at the one-hour mark. So after day 7 you can’t tell how the device was doing at the top of the hour versus the half hour because you have three values – the minimum, the maximum, and an average. The minimum and maximum are more interesting for things like latency and speed, but the average tells us, on average, how often it responded to ping. After 30 days, those averages to go to an entire day.

So, make sure you’ve set your retention settings to support the maximum reporting requirement you’ll need. It’s not fun to have to generate a report and then realize after you need it that you don’t have enough data with the right precision.

You can find these settings under Settings > All Settings > Thresholds & Polling > Polling Settings.

Customize your data retention

3. Create a DeviceType Custom Property

Because SolarWinds is capable of monitoring so many types of devices from servers to routers to fiber switches to UPSs, it’s important to be able to categorize them. Also because SolarWinds treats devices very generically (with some exceptions, like Cisco UCS and VMware) it doesn’t hazard a guess as to what “flavor” of device you’ve added.

Since that information isn’t assumed by the system, it’s good to have a custom property that can be set on each device or cluster of devices as they’re imported into the system.

Some good values for that custom property follow:

  • Router
  • Switch
  • Power Supply
  • Printer
  • Server
  • Environmental Sensor

Armed with this custom property you can use it to customize the rest of your SolarWinds configuration. For example:

  • View all your routers and switches only
  • Limit an alert about your UPS devices to your facilities distribution list only

4. Create a THWACK Account

There’s nothing worse than working with software in a vacuum. While you’re working with SolarWinds software you’re going to have questions. The resources I’m going to want to find immediately upon being put in charge of a new install are the following:

  • Where can I learn more about this software?
  • If I have a problem, who can I ask?
THWACK IT Community

THWACK has you covered. There are forums for each of the products and they’re well attended by SolarWinds administrators, hobbyists, and SolarWinds’s very own MVPs.

Besides product-specific forums they have special areas for the API and labs for reporting and alerts because they got enough traffic that they were split off from the main content. Links are below to check those out, but don’t forget to create an account – that’s the only way you can contribute and take part in the conversation.

Important THWACK Links

5. Install SolarWinds SDK

Programming isn’t for everyone, that’s true. So it’s possible that you might have read about the SolarWinds SDK (software development kit) and passed right over it. What you didn’t see is that this software comes with an extra sharp tool called SWQL Studio (pronounced swickle).

If you’ve ever used SQL Studio from Microsoft to interact with a database, this utility won’t be too hard to get into your toolbelt. These are the major differences:

Entities vs. Tables

This is partly a naming convention and partly a functional one. In the database your node information is in an appropriately-named table (okay, it’s actually a view) called Nodes. In SWQL, the same information can be found in the entity called Orion.Nodes. Your volume information is in a table called Volumes. In SWQL, it’s in Orion.Volumes. That’s great – are we just prefixing everything with Orion? No, but there is a search, so you can find the entity easily.

Select Star From Table

When you want to get to know a table and its contents, the DB wrangler in us all taps out the usual query:

SolarWinds Administrator Select Star From Table

Not to worry, you can still double-click on the entity and get a tidy SELECT query with all the fields. From there you can whittle it down to just the fields you want.

Select Star From Table Query

JOIN vs. Navigation Properties

This is probably the very coolest difference between being thrown into a database you don’t know and working with SolarWinds data.

Usually you would have to know that a) two tables are related and b) that there’s a key between them that you can use to establish a relationship by way of some type of JOIN. With SWQL they curate these relationships for you.

They’re represented here by these chain-link icons. The string to the left is the navigation and the bit in the parentheses is the actual entity. There may be some slight difference between them as is the case with ResponseTimeHistory (navigation property) and Orion.ResponseTime (the entity itself).

JOINs vs. Navigation Properties

There will be a follow-up article that goes into using SWQL in more detail, but here’s a taste – we are working with the Orion.Nodes entity and we want to reach into the Orion.NPM.Interfaces entity only for the purpose of taking the InterfaceName field from that entity.

Orion.NPM.Interfaces entity


In this blog post you learned how to:

  1. Get automatic email SMTP notifications when things go wrong
  2. Make sure your reports have the data that they need
  3. Add extra information specific to your environment
  4. Get help when you need it (and share what you’ve learned) on THWACK
  5. Get the data back out of SolarWinds using SWQL

I hope you’ve enjoyed this blog and you’ll stay tuned for others that are coming soon.

Steven Klassen - Programmer Analyst

Steven Klassen

Programmer Analyst

Steven Klassen is a principal developer for Loop1, designing and developing SolarWinds integrations primarily in Go, PowerShell, Python, and JavaScript. He has more than 20 years of experience in software automation and technical training. He teaches public training classes on database design, queries in SQL and SWQL, and programming in Go, Python, and PowerShell. Steven developed and teaches the Loop1, 1-day virtual Solar Flare coder camp class.

Steven enjoys learning new programming languages, technologies, APIs, and just about anything else. His very favorite thing to do is breaking down a technical concept into easy-to-learn bits by spinning analogies. Steven likes to listen to music while he works, and that usually consists of whatever the Biebs has put out recently along with anything and everything written between ’84 and ’95.

Log4Shell Vulnerability covered by Runecast - Request a Vulnerability Assessment Request Assessmentx