Setting Up Kube-Prometheus-Stack on Kubernetes Services

This setup will install the application stack of Prometheus and Grafana together on a K8 Kubernetes Cluster.
Great Dashboards comes out of the box with the application-stack.

Prerequisites:

• Kind-Cluster (or others cluster) with Ingress-controller (HTTPS or DNS challenge) configured
  If you don’t want use an ingress you can access over a port-forwarding to load the grafan gui!

# Update the Helm repositories
helm repo update

# Add the Kube Prometheus Stack Helm repository
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

create config file: kube-prometheus-values.yaml

# Sample kube-prometheus-values.yaml
prometheus:
  enabled: true

alertmanager:
  enabled: true

grafana:
  enabled: true

Run installation:

# Create the "monitoring" namespace
kubectl create namespace monitoring

# Deploy the Kube Prometheus Stack
helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack -n monitoring -f kube-prometheus-values.yaml

# Deploy the nginx ingress
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace monitoring \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz 

Ingress config standard option #1 – Create config file: grafana-ingress.yaml

# Sample grafana-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: monitoring
spec:
  rules:
    - host: *
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kube-prometheus-stack-grafana
                port:
                  number: 80

If you want ingress with lets-encrypt and bot urls for Grafana and Prometheus then use option #2, this example:
For this example i have created the according DNS entry´s on Hetzner before with the according IP Address of the VM vsphere machine!

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: monitoring
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-staging"
    ### Uncomment for production usage
#    cert-manager.io/cluster-issuer: "letsencrypt-production"
spec:
  tls:
  - hosts:
    - laborgraf.youredomain.de
    secretName: laborgraf.youredomain.de
  rules:
  - host: laborgraf.youredomain.de
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kube-prometheus-stack-grafana
            port:
              number: 80

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prometheus-ingress
  namespace: monitoring
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-staging"
    ### Uncomment for production usage
#    cert-manager.io/cluster-issuer: "letsencrypt-production"
spec:
  tls:
  - hosts:
    - laborprom.youredomain.de
    secretName: laborprom.youredomain.de
  rules:
  - host: laborprom.youredomain.de
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kube-prometheus-stack-prometheus
            port:
              number: 9090

run kubectl apply -f grafana-ingress.yaml

1. Access the Grafana dashboard using the external IP or hostname / url configured for the Ingress.
2. Log in to Grafana using the initial username and password.
   ”admin / prom-operator”
3. Navigate to the Configuration section and click on “Add data source”.
4. Select “Prometheus” as the data source type.
5. Configure the URL of the Prometheus server. By default, it should be `http://prometheus-operator:9090`.
6. Save and Test the connection. Upon successful connection you can see a success message.
7. Now you can navigate to the Dashboard section and check the available dashboards.
8. Change the initial Password !