Symptoms
-
The following is shown by Grafana:
Firing Value: [no value] Labels: - alertname = DatasourceError - rule_uid = XxXxXxXx - rulename = Partition "/var" utilization - sourceKey = hdd_util_var Annotations: - message = The disk space is almost full! Source: /modules/grafana/service/alerting/grafana/XxXxXxXx/view Silence: /modules/grafana/service/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceError&matcher=rule_uid%XxXxXxXx&matcher=rulename%3DPartition+%22%2Fvar%22+utilization&matcher=sourceKey%3Dhdd_utilvar Dashboard: /modules/grafana/service/d/monitoringhdd Panel: /modules/grafana/service/d/monitoring_hdd?viewPanel=4
*Firing* Value: [no value] Labels: - alertname = DatasourceError - rule_uid = XxXxXxXx - rulename = nginx memory usage - sourceKey = memory_usage__webproxy Annotations: - message = The memory usage status is critical! Source: /modules/grafana/service/alerting/grafana/XxXxXxXx/view Silence: /modules/grafana/service/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceError&matcher=rule_uid%XxXxXxXx&matcher=rulename%3Dnginx+memory+usage&matcher=sourceKey%3Dmemory_usage__webproxy Dashboard: /modules/grafana/service/d/monitoring__services Panel: /modules/grafana/service/d/monitoring__services?viewPanel=7
The current value is . -
Plesk notifications show messages about exceeded thresholds which were not reached
-
The following message can be found in
/var/log/grafana/grafana.log
:logger=plugin.manager t=2022-06-22T13:46:52.610714606+01:00 level=error msg="Could not start plugin" pluginId=grafana-simple-json-backend-datasource err="Incompatible API version with plugin. Plugin version: 1, Client versions: [2]"
logger=ngalert uid=XxXxXxX org=1 t=2022-06-22T04:39:50.041885056+01:00 level=error msg="failed to fetch alert rule" err="database is locked"
Cause
Unsupported Grafana version is installed.
Resolution
For Plesk on RHEL based Linux (AlmaLinux/RockyLinux/CentOS)
-
Connect to the server via SSH
-
Downgrade version of the the grafana package to the current one:
# yumdb search from_repo plesk-ext-grafana
Loaded plugins: fastestmirror
grafana-8.3.6-1.x86_64
from_repo = plesk-ext-grafana# rpm -e --nodeps grafana.x86_64 9.0.0-1
# yum install grafana-8.3.6-1.x86_64
-
Restart grafana:
# service grafana-server restart
For Plesk on Debian based Linux (Debian/Ubuntu)
-
Connect to the server via SSH
-
Find the file containing non Plesk Grafana repository with this command:
# grep -r graf /etc/apt/
/etc/apt/sources.list:deb https://packages.grafana.com/oss/deb stable main
/etc/apt/sources.list.d/plesk-ext-grafana.list:deb https://autoinstall.plesk.com/grafana/deb stable main -
Remove the line containing "packages.grafana.com" from the file found in step 2. (
/etc/apt/sources.list
) -
Install supported Grafana version:
# apt-get install grafana=8.3.6
-
Restart Grafana:
# service grafana-server restart