Technical Audit for Large Websites: Best Practices and Methodologies

Big sites, which have thousands of potential pages, can have so many things go wrong with them. You need to have intense diligence when working on massive sites because there’s so much to review. Naturally, these mega sites have likely had multiple SEO experts work on them and small issues were missed.

Over time, these small issues compound. Then, someone like me comes along and has to do a major technical audit.

I’ve had to do some pretty major audits in the past, and I’m sure I’ll have to do them again in the future. Before I dive into how to do an audit, I want to stress that you may need to delegate tasks. You can easily find thousands of errors on a site that would take weeks to correct on your own.

Delegating tasks can help make fixing these sites easier, as long as you make sure everyone is on the same page.

Story Time: Find out who is making SEO changes

I remember one situation where I ran an audit and began fixing technical problems on the site. However, another team was working on the SEO and was making more errors than I could fix. After scratching my head and trying to figure out what was going on, I asked the marketing team who was working on the site. It turned out, they were working with multiple optimization companies.

I spent an additional 10 hours fixing errors that kept popping up on the site because I didn’t find out who else was working on optimizing the site.

Moral of the story: When you do a technical audit, be sure that you know if others are doing SEO on the site and discuss the changes you’re making.

101 Guide on Starting a Technical Audit on Big Sites

Auditing is a lot of work. While you can do everything manually, you’re not going to be as efficient as running a crawler. I use Screaming Frog and suggest you do too. You can also use other crawlers to monitor:

• Broken links
• Redirects
• Slow pages
• Meta tags
• Internal links
• Much more

I recommend starting your audit with the most impactful issues that technical SEO involves. A broken image is far less of a concern than a redirect error, causing lost traffic on one of the site’s highest revenue-generating pages.

While your crawler is running, you’ll want to take the time to look at the robots.txt.

Open up the robots.txt file and analyze it for any lines that have “disallow” in them.

Now, once your crawler is finished running, you’ll want to go through the following issues in this order and fix them:

1. Redirects. Do you have any broken redirects, redirect loops or chains? You need to fix these issues first because they’re going to have a drastic impact on your site’s rankings. Go through the list on Screaming Frog to identify redirect issues. Now, go through these errors and fix the redirects based on the page’s priority. High-trafficked pages are the first you need to fix.
2. Slow pages. Run a few pages through Google PageSpeed and fix any site speed issues. Caching, large images, CSS and JavaScript file sizes and other issues can normally be optimized to speed up the site. I recommend starting with caching pages and databases and then working on setting up a CDN. You might need to recommend higher resource servers and hosting if none of the recommendations work to speed up the site.
3. Canonicalized Tags. Review the canonicalized tags to learn where the tags are pointed to and ensure that they’re pointing to relevant pages.
4. Meta Data. Check the meta titles and descriptions for each page to ensure they exist and are optimized for search engine optimization. You’ll want to optimize all of these pages to enhance their SEO as much as possible.
5. Internal Links. The site’s internal linking must be done with precision. Google loves when you create content in clusters and interlinking relevant content will help provide a boost to the site’s rankings. If you find broken internal links on the site, these are the links that you’ll want to fix first.
6. Images. Broken image links will slow up your site and cause the site’s user experience to decline. Identify these images, replace them and ensure that the site’s images all have proper alt tag optimization in place. I also recommend setting a CDN up to speed up image delivery.
7. Duplicate Content. Screaming Frog will detect duplicate content that you need to fix up. You can go through these pages to make changes to any duplicate content that you have or redirect the duplicate page to the main one.

Re-test and Repeat

Next comes the fun part. Once you’re done going through all of these points, you’ll want to run your crawler again and repeat the steps above. The goal is to remedy as many errors and issues with the site as possible.

It’s a timely process, but it’s one that needs to be made for technical search engine optimization.

If you work through these seven points, you’ll have performed a rather thorough technical SEO audit. However, I do recommend working on the site’s SEO hygiene as part of a technical audit, too.

Introduce SEO Hygiene to the Site

SEO hygiene is the “icing on the cake.” It’s going to position the site to maintain its technological advantage for years to come. What this means is that you’ll put people in charge of:

• Analyzing the site’s code and cleaning up errors, issues with sections of the site not loading and rendering issues.
• Content and ownership basics. Multiple teams are creating content on these larger sites, and it’s crucial to have systems in place to track keywords, content and ideas. You don’t want multiple pages on your site competing for the same keywords and cannibalizing your efforts.
• Redirect monitoring. I recommend creating a central database for redirects so that loops, chains and other issues are less of a problem in the future.
• Quality assurance measures should be in place to reduce errors. Central logs and procedures should be in place to reduce error risks.

Website architecture also needs to be considered with SEO hygiene to ensure that your site is as easy as possible to crawl and index by search engines.

Technical audits are a lot of prep work and reviewing reports repeatedly. You’ll become more familiar with the site than the developers when you’re done with your audit, but the results can be amazing.

A site that focuses on technical SEO has an advantage when trying to rank than a domain that isn’t.