When you encounter a “Couldn’t fetch” error for your sitemap, it generally means that the search engine (like Google) is having trouble accessing or retrieving your sitemap file. Here’s a step-by-step guide to troubleshoot and fix this issue:
1. Check Sitemap URL
Ensure that the URL you provided for the sitemap is correct. Common URLs for sitemaps typically follow standard naming conventions and paths. Here are some frequently used formats for WordPress Sites:
https://www.yourdomain.com/sitemap_index.xml(often used with SEO plugins like Yoast)https://www.yourdomain.com/sitemap.xml(default location for various other plugins)
Note: Replace ‘yourdomain’ in the above URL’s with your domain name.
For Example, the Sitemap URL for myWebsite is https://socalledbtech.com/sitemap_index.xml which uses Yoast SEO.
2. Verify Sitemap Accessibility
Open your sitemap URL in a web browser to check if it loads correctly. If it doesn’t, the issue might be with the sitemap file itself or server settings.
3. Confirm Sitemap Format
Verify that your sitemap follows the correct format. XML sitemaps should follow the standard XML format. You can validate your sitemap using online tools like XML Sitemap Validator.
How to Use:
- Go to the XML Sitemap Validator page.
- Enter the URL of your XML sitemap and click on VALIDATE SITEMAP.
- Review the validation results for any errors or warnings.
4. Check Server Configuration
- Firewall or Security Plugins: Ensure that your server’s firewall or security plugins aren’t blocking search engine bots from accessing the sitemap.
- Robots.txt File: Make sure your
robots.txtfile is not disallowing access to the sitemap. Yourrobots.txtshould not have aDisallowrule that blocks the sitemap URL.
Note: The robots.txt file is typically located at the root of your domain, e.g., https://www.yourdomain.com/robots.txt. You can access it by navigating to this URL in a web browser.
Correct Example: You can refer to the below image for my website robots.txt file which does not have a Disallow rule that blocks the sitemap URL.
Incorrect Example: The below code contains Disallow rule that blocks the sitemap URL.
User-agent: *
Disallow: /sitemap.xml
5. Review HTTP Status Codes
To ensure that search engines can properly access and index your sitemap, you need to verify that the sitemap URL returns a 200 OK HTTP status code. If your sitemap returns a 404 Not Found or 403 Forbidden status, it indicates issues that need to be addressed. Here’s how to check and address these status codes:
Check Status Code
Using a Browser
- Enter the sitemap URL (e.g.,
https://www.example.com/sitemap.xml). - Open Developer Tools (press
F12orCtrl+Shift+I). - Go to the “Network” tab and reload the page.
- Check the status code (should be
200 OK).
Using Online Tools
- Enter your sitemap URL in HTTP Status Code Checker
- Click “Check” to see the status code.
Addressing Common Status Codes
- 404 Not Found: Verify the URL and ensure the file exists.
- 403 Forbidden: Check file permissions and server access controls.
- 500 Internal Server Error: Review server logs and configurations.
Verify Changes
- Recheck: Ensure the status code is
200 OK. - Resubmit: Update your sitemap in Google Search Console or other tools.
Maintaining a 200 OK status ensures your sitemap is accessible to search engines.
6. Check for Sitemap Errors
If you are using CMS plugins (e.g., Yoast SEO for WordPress), make sure the plugin is properly configured and updated. Sometimes, plugin issues or outdated versions can cause problems.
7. Review Server Logs
Check your server’s error logs for any issues that might be preventing the sitemap from being fetched.
8. Update and Resubmit Sitemap
After making any necessary changes, update your sitemap in the search engine console (e.g., Google Search Console) and request a re-crawl.
9. Consider Sitemap Size
Ensure that your sitemap isn’t too large. Search engines might have trouble fetching very large sitemaps. If necessary, split large sitemaps into smaller ones.
10. Clear Cache
If you’re using a caching plugin or service, clear the cache to make sure the search engine can access the most recent version of your sitemap.
11. Test with Fetch as Google Tool
In Google Search Console, use the “URL Inspection” tool to see how Googlebot fetches your sitemap. This can provide insights into any issues.
12. Check for Recent Changes
If you recently made changes to your website or server, double-check those changes to ensure they didn’t inadvertently cause issues with the sitemap.
By following these steps, you should be able to diagnose and resolve the issue with your sitemap not being fetched. If problems persist, consulting with a developer or reaching out to your hosting provider for assistance may be necessary.