Post-upgrade Application Checks

Once the Kubernetes upgrade is complete, it's important to validate that applications are functioning as expected. This checklist focuses on verifying critical application behavior, observability, and scheduled tasks.

---

1. Functional Validation

Perform functional validation of applications

Test key features of each application to ensure nothing broke during the upgrade.

Steps:

1 Manually validate UI responsiveness and user flows (e.g. login, navigation, form submission).

2 Test application-level authentication and authorization mechanisms.

3 Trigger any known API integrations (e.g. payment gateways, external APIs, webhooks) and ensure they respond correctly.

4 Validate database reads/writes where applicable to confirm full backend connectivity.

---

2. Observability

Validate application logs, metrics, and alerts

Post-upgrade, it's important to verify that all observability pipelines are intact.

Steps:

1 Check application logs for runtime errors or new warnings introduced after the upgrade.

2 Review dashboards (e.g. Grafana) to ensure metrics are updating in real time.

3 Confirm alerting rules are still in place and functional by simulating a known failure condition if possible.

4 Ensure log forwarders (e.g. FluentBit, Logstash) are still collecting from your application pods.

---

3. Scheduled Tasks

Verify CronJobs

Make sure scheduled tasks (like backups, report generation, or syncing jobs) are still operating correctly.

Steps:

1 List all CronJobs and check their lastScheduleTime:

kubectl get cronjobs -A

2 Inspect recent CronJob pod runs:

kubectl get pods --sort-by=.status.startTime -A | grep cronjob

3 Check logs of recent jobs to confirm successful execution:

kubectl logs <job-pod-name>

If any jobs have failed or are not running on schedule, review their schedule, suspend field, and successfulJobsHistoryLimit.