Assume the Pods are running fine but you are getting no response when you try to access the Service.
1. Check the does the Service exist using the below command,
kubectl get svc <service name>
If the service is not present, create the service with appropriate configuration.
If the service is available and still unable to access it then check any NetworkPolicy ingress rules affecting the target Pods.
2. Check if the service work by DNS name.
One of the most common ways that clients consume a Service is through a DNS name.
Try the below from a Pod with in the same Namespace,
nslookup <service name>
if the above command fails then Pod and Service are in different NameSpaces.Try a namespace -qualified name again from within Pod,
nslookup <service name>.<namespace name>
If this works, you will need to modify your app to use a cross-namespace name or run your app and Service in the same Namespace. If this still fails, try a fully qualified name.
nslookup <service name>.<namespace name>.svc.cluster.local
If you are able to do a fully qualified name lookup but not relative one, you need to check that your /etc/resolv.conf file in your Pod is correct.
Try the below from within a Pod,
cat /etc/resolv.conf
You should see something like below,
nameserver 10.0.0.10
search default.svc.cluster.local svc.cluster.local cluster.local example.com
options ndots:5
The nameserver line must indicate your cluster's DNS Service. This is passed into kubelet with the --cluster-dns flag.
The search line must include an appropriate suffix for you to find the Service name. In this case it is looking for Services in the local Namespace ("default.svc.cluster.local"), Services in all Namespaces ("svc.cluster.local"), and lastly for names in the cluster ("cluster.local").
The options line must set ndots high enough that your DNS client library considers search paths at all. Kubernetes sets this to 5 by default, which is high enough to cover all of the DNS names it generates.
3. Does any Service work by DNS name?
If the above all fails, DNS lookups are not working for your service.
The Kubernetes master Service should always work. From within a Pod:
nslookup kubernetes.default
If this fails, Please check the kube-proxy configuration and also debug the DNS Service.
4.Does the Service work by IP?
Assuming you have confirmed that DNS works, the next thing to test is whether your Service works by its IP address. From a Pod in your cluster, access the Service’s IP
5. Is the Service defined correctly?
If the above one fails, check that your Service is correct and matches your Pod’s port using the below command,
kubectl get service <service name> -o json
Below is the check list,
- Is the Service port you are trying to access listed in
spec.ports[]? - Is the
targetPortcorrect for your Pods (some Pods use a different port than the Service)? - If you meant to use a numeric port, is it a number (9376) or a string “9376”?
- If you meant to use a named port, do your Pods expose a port with the same name?
- Is the port’s
protocolcorrect for your Pods?
6. Does the Service have any Endpoints?
If you are here means, you have confirmed the Service is defined correctly and is resolved by DNS.
Now lets check the Pods are actually being selected by the Service.
Checking the endpoints for Service called hostnames,
kubectl get endpoints hostnames
NAME ENDPOINTS
hostnames 10.244.0.5:9376,10.244.0.6:9376,10.244.0.7:9376
This confirms that the endpoints controller has found the correct Pods for your Service. If the ENDPOINTS column is <none>, you should check that the spec.selector field of your Service actually selects for metadata.labels values on your Pods.
