Too many open files debug in k8s

ulimit -n 
 Output : 1048576 
 kubelet limit 
 systemctl show kubelet | grep NOFILE
 
 Output : 
 LimitNOFILE=524288
LimitNOFILESoft=1024
 
 systemd global 
 systemctl show --property=DefaultLimitNOFILE
 
 Output: DefaultLimitNOFILE=524288 
 container runtime 
 systemctl show containerd | grep NOFILE
 
 Output: 
 LimitNOFILE=infinity
LimitNOFILESoft=infinity
 
 Socket summary 
 ss -s 
 Total: 795
TCP: 2101 (estab 22, closed 2066, orphaned 0, timewait 20)

Transport Total IP IPv6
RAW 1 0 1 
UDP 8 5 3 
TCP 35 27 8 
INET 44 32 12 
FRAG 0 0 0 

 
 System-wide (all processes) 
 cat /proc/sys/fs/file-nr
Output: 
 17784 0 2097152
 
 Kubelet FD usage 
 pidof kubelet
ls /proc/$(pidof kubelet)/fd | wc -l 
 Output: 1953 
 Edit Container runtime 
 sudo systemctl edit containerd 
 Per-pod socket usage (ADVANCED) 
 crictl ps
crictl inspect <container_id> | grep pid
 
 Correct fix for kubelet (DO THIS) 
 sudo systemctl edit kubelet
 
 [Service]
 LimitNOFILE=524288
 
 sudo systemctl daemon-reload
sudo systemctl restart kubelet