I’ve been building, fixing, and optimizing websites for nearly 15 years. In that time, I’ve seen people do some crazy things to their websites; but, more often than not, it’s the simple SEO mistakes that do the most harm. In this post I’ll go over the top things I’ve encountered over the years that do more harm than good.
Robot.txt File Blocking Parts of the Website
Having a few bad lines in your robots.txt file can make the traffic to website plummet to zero in a matter of days. The most common way this happens is when the file is copied from a development server to live server, and nobody remembers the robots.txt until it’s too late. The culprit is usually two lines of code that look like this: User-agent: * Disallow: / In reality, using the disallow to block parts of your site should only be used on parts of the site that have confidential data, which should also be protected by a password. Using a robots file to block things like shopping carts, wish list accounts, customer service, about us, or other “secondary” pages isn’t a good idea. The fact is modern search engines “like” to see those kind of files and information. It helps them know you are a legitimate website. You also want to avoid blocking things like CSS and JavaScript files. Recent evidence suggests that Google is looking at your CSS and JS to make sure you aren’t doing anything deceptive. If you have a page that you want the search engines to see, but you don’t want to show up in search results, use the Noindex directive instead: User-agent: * Noindex: /
Use Standard URLs
Another really common mistake people make is creating URLs that aren’t search engine friendly. Search engines have gotten a lot more sophisticated and can handle much more complicated URL’s than they could in the past, but that doesn’t mean you can do whatever you want. Here are some basic guidelines:
- Use standard file types or extensions (i.e. html, asp, jsp, php)
- Shorter is always better than longer
- Words are better than numbers
- Use delimiters between words
- Avoid using non standard characters (i.e. @, *, $)
- Keep parameters to a minimum or avoid entirely if possible
If you are already making a mistake and if the search engines are ranking your pages and sending you traffic, don’t change it for the sake of changing it. However, resist the temptation to “push it” to see what you can get away with. Don’t make things worse than they already are.
Use Standard Layouts
Modern search engines can scan your pages and separate your template, header, navigation, sidebars, and footer from your main content–as long as you don’t throw them too big a curveball. I once worked on a project where they did a conversions study that gave them a 15% increase in conversions when they put the form in the middle of the page. The problem was the “content” part was swapped out and put in the sidebar. The search engines weren’t able to figure it out and traffic dropped by 80% over a period of one month. Once we put the content back in the center, traffic returned to normal. Keep things easy for the search engines to figure out and don’t try to be too “different” or too “bleeding edge.”
Get your free seo analysis today
Duplicate Titles and Descriptions
Another common mistake websites make is to have identical or nearly identical titles and/or meta descriptions on multiple pages, or even across the site as a whole. Your page’s title element is one of your primary on-page factors use to rank a page. If you use identical or nearly identical titles on multiple pages, Google will pick one and ignore the others. While meta descriptions aren’t a factor in ranking, having identical or nearly identical descriptions on multiple pages isn’t a quality signal to send to the search engines. To be on the safe side, you want to avoid repeating headings or blocks of text on multiple pages of your site if you can avoid it.
Errors and Soft 404 Pages
When a page is requested from a web server, there is a behind the scenes response code that is sent that says “this page is ok,” “this page has moved,” or “this page doesn’t exist.” When a page serves normally, the response code is 200; when the page doesn’t exist, the response code is 404. If someone requests a page that isn’t there, but your server sends a 200 response code, this is what the search engines call a “soft 404.” Search engines have a hard time with this, especially if your website has a lot of them. If you have this problem, correct it as soon as possible, and try to avoid it in the future.
Canonicalization
Canonicalization is a really technical term that simply means making sure there is only one URL for any given page. For example, if each of these URLs serves the same page on your website, you have a canonicalization problem. example.com www.example.com www.example.com/ www.example.com/index.php You want to make sure your site either has the “www” or doesn’t. If someone requests the “wrong” version, you want to redirect them. You want to be consistent about the use of a trailing slash “/”; in most cases, it’s a good idea to force it to be on the end. Lastly, you want to be consistent about needing/showing files at the end, It’s usually best to leave them off if you can, but as long as you are consistent about always showing it or never showing it, you won’t have a problem. Those are some common, easy to solve, high level mistakes that people are likely to make. Correcting as many as you can and making sure other don’t creep in will help keep your site healthy and easy to understand in the eyes of the search engines.
