What is Grafana, and what is Grafana monitoring? When Torkel Ödegaard took the decision in 2013 to fork Kibana, it heralded the beginning of the Grafana project, turning its forebear into a graph-focused and time-series dashboard tool. Ödegaard’s intention and guiding principle in creating it was to realize a dashboard that was dedicated to clarifying data by removing distractions and being a user-friendly and elegant solution. He seems to have succeeded in achieving his aims because half a million active installations don’t lie. These days you can find Grafana dashboards everywhere. Even if you’re unfamiliar with them, you may have seen one recently if you’re a fan of watching SpaceX launches!
Grafana Features
Grafana monitoring is achieved using panels. The basic building block for visualization in Grafana is the panel, and that panel can contain a graph, a Singlestat, a table, a heatmap, and freetext, and it can also integrate with both proprietary and community-created plugins too (like a clock or world map, for instance). Users are free to customize the style and format of each panel, and they can drag, drop, and resize them as they wish to create the ultimate visualization to suit their needs.
So, a Dashboard is a collection of panels, each of which holds a set of variables (things like sensor name, application, and server). These panels are arranged in a grid on the Grafana dashboard, and the user can change the data being scrutinized by switching variables, and that could be data from two different servers, for instance. Although the flexibility to customize views is one of the strongest Grafana features, users can just as easily pick up one of many ready-made dashboards to handle different data types and sources. Grafana’s large community of users and contributors has already created lots of them.
Grafana monitoring can include annotations to show particular events across the panels. Adding an annotation is achieved by putting in a custom request to Elasticsearch. Doing so makes this show up on the graph as a vertical red line. Hovering over an annotation then gives you an event description and tags (for instance) so you can track when the server returns a 5xx error code or when there’s a system restart, for instance. This makes it particularly easy to investigate system behavior and to track particular events and their consequences in an application.
Grafana Dashboard and Custom Web Apps
Grafana Dashboard and Graylog
Graylog can be used for the storage and management of web app logs and the monitoring of their performance, not just in production but also during the development stage. Grafana monitoring expresses these logs visually, to make analyzing the system more straightforward. You could legitimately describe Grafana as a web application load and performance user interface as well as a visitor flow tool. Graylog and Grafana work well together but there was no special effort made to integrate them. Graylog stores all log data in Elasticsearch, one of Grafana’s data sources, so it was easy enough to use one of the Elasticsearch indexes to connect Grafana to Graylog.
Visualizing Web Application Metrics in Grafana Dashboard
Grafana isn’t interested in error notifications or pure text logs because its main job is visualizing the data in tables, charts, and graphs. The developers created a custom Django module to track data on each web/worker request and processed response, not just reporting whether it succeeded or failed, but also providing a set of general and project-specific structured fields, including:
- app version
- ID for each unique request
- response time and status
- error code (if applicable)
- IP address the request was sent from
- user details (username for registered user’s email address, role, permissions)
- device details
Django pushes these custom-structured analytical records into Graylog, which stores them in a different stream. Although Graylog dashboards can visualize this kind of data natively, they aren’t as adept at examining Grafana’s, so Grafana was adapted to visualize this analytical data. It can track application performance and load in real-time as well as retrospectively.
Top Features of Grafana
Grafana Labs is the name of the company that was created to push for the adoption of Grafana and to turn the project into a viable business. Whether you’re new to the Grafana dashboard or not, there’s a chance you may not be aware of the features that have been added to it both by Grafana Labs and its enthusiastic community.
Let’s take a look at some of the best ones:
Dashboard templating
This is one Grafana feature that’s really useful. It allows users to create a dashboard setup to suit their every need. And these templates don’t come with hardcoded values, which means that if you have a test server and a production server, the same dashboard will work with both. Templating lets you examine data at every level from the macro to the micro, so you can start with a whole country, for example, then drill down to a particular region, and keep going as far as granularity allows. These dashboards are then shareable with everyone from teams throughout your organization to the whole community.
Provisioning
It may be easy enough to set up a single dashboard with some clicking, dragging, and dropping, but some users need even more simplicity in a way that scales. So, Grafana features provisioning so you can automate setup using a script. Anything can be scripted in Grafana. For instance, when you want to create a new Kubernetes cluster, you can have Grafana automatically help with a script that already has the right server, IP address, and data sources set up and locked. This is also a way to control lots of dashboards.
Annotations
This Grafana feature lets you mark graphs, which is particularly helpful if you need to correlate data when something misbehaves. You can control-click and type on a graph to create your annotations manually, or data can be fetched from any source to populate them. (You can see an example of this in the way that Wikimedia uses annotations on its public Grafana dashboard.) A good use case would be automatically creating annotations at the time of releases. If you were to start seeing errors a little while after a new release, you could go back to your annotations and check if the errors correlate. This kind of automation is possible with the Grafana HTTP API. Lots of Grafana’s biggest customers use it for a wide range of tasks, with a common one being to set up databases and add users. This is an alternative to provisioning for automation, and there’s more you can do with it. For instance, DigitalOcean’s team used the API to include a snapshot feature that helps them to review dashboards.
Kiosk mode and playlists
Playlists are great for ‘rolling coverage’. You select which Grafana dashboards you would like to display on a monitor or TV, and it can cycle through them throughout the day. Kiosk mode lets you only show the user interface elements that you need in view-only mode. Useful tip: The Grafana Kiosk utility handles logins, switching to kiosk mode, and opening a playlist, so if a TV you want to use has no keyboard you can still set it up without hassle.
Custom plugins
You can extend Grafana’s functionality with plugins that offer extra tools, visualizations, and more. Popular examples include Worldmap Panel (which superimposes data on a map), Zabbix (which integrates with Zabbix metrics), and Influx Admin Panel (which enables database creation or lets you add users). These are just a couple of examples and there are many others besides them. Write a little code and Grafana can visualize anything that produces a timestamp. Also, Grafana Enterprise customers can access additional plugins that facilitate integrations with Datadog, New Relic, Splunk, and others.
Alerting and alert hooks
Grafana alerts can be sent through several different notifiers, including email, PagerDuty, or Slack or texts. If these don’t work for you, it’s easy enough to code alert hooks that create different notifiers.
Teams and permissions
Where an organization has one instance of Grafana and several teams, they usually like to have the option to enforce some dashboard segregation. It used to be the case that this wasn’t possible because Grafana automatically made everyone’s dashboards accessible to everyone else. The later edition of multi-tenant mode meant that users could switch organizations but couldn’t share dashboards. Some judicious hacks could enable both, so Grafana created an easier route to achieving this. It’s now possible to create a team of users and then assign permissions on folders, Grafana dashboards, and so on, right down to the data source level for Grafana Enterprise users.
SQL data sources
Grafana natively supports SQL, which helps you to graph any kind of data that might be held in an SQL database. High-end users are doing lots of interesting things with SQL data sources, including building business dashboards that “make sense to your boss’s boss,” (as one team put it).
Monitoring your monitoring
If you take your monitoring seriously enough to want to monitor your monitoring, Grafana features its own Prometheus HTTP endpoint that can be scraped by Prometheus, making it fairly easy to get statistics and dashboards. Once the enterprise version is up and running you’ll be able to get Google Analytics-style data access, so you can find out just how much CPU your Grafana is chewing through or how much time alerting takes.
Authentication
Grafana supports LDAP and OA and other authentication styles, and lets you map users to organizations. With Grafana Enterprise, it’s also possible to map users to teams: so if your organization uses its own authentication system, Grafana lets you map teams in your in-house systems to teams in Grafana, which automatically gives team members access to their own designated Grafana dashboards.
Grafana on Plesk
Plesk provides Grafana integration using Grafana Extension, the premier open-source software for time-series analytics. Grafana can turn all kinds of data into all kinds of visually appealing graphs and dashboards that can be customized in endless ways.
You can use Grafana to:
- visualize data pulled from default sources, which the Plesk team can integrate for you. The default source at present is the Advanced Monitoring extension, which gathers metrics on server health.
as a more experienced Grafana user, you can pull data from any source that will integrate with Grafana. If you’d like to do this, just give the appropriate permission to the Grafana administrator.
with
No comment yet, add your voice below!