Microsys Blog

Just a quick update for those following development of our sitemap generator program.

The new version 2.0.2 contains improved support for video sitemaps and new support for code sitemaps. While support for both sitemap file kinds may still have a few rough edges, you can now create complete video sitemaps and code sitemaps for most websites in a fully automated way

Should you encounter any issue or have feedback, please let us know. We will be ironing out any issues in the next release. We also intend to improve our sitemap generator tutorials section soon including those tutorials covering the more exotic sitemap file kinds

The last few days I have been working on creating a database covering common questions I have received about Google Webmaster Tools, XML Sitemaps, HTML Sitemaps and the A1 Sitemap Generator program itself of course

The FAQ covers almost everything including how to diagnose and solve crawling errors, handle websites on unstable webservers, scan large websites, questions about XML Sitemaps Protocol, weird Google Webmaster Tool issues, how to configure A1 Sitemap Generator etc.

If you have ever had any questions regarding sitemaps, I highly recommend checking our
sitemapper tutorials and the new sitemap questions and answers database

Sometimes I get asked about priority values in XML sitemaps created by A1 Sitemap Generator. You can choose one of the following ways to handle priority values:

• Have XML sitemap priority values calculated based on all internal links within the website. There is a setting to prevent “extreme values” to be used, e.g. 0.0. This is the default configuration.
• Override priority and set same value for all pages, e.g. 0.5.
• Leave out priority values in total. This is probably preferred to using the same value for all pages.

For more information about how priority values are calculated, see our XML sitemaps priority value article.

The new release of our sitemap generator tool has gone through months of development with special focus on increasing website crawler performance. However, that is not all Lots of feedback (thank you) has helped me to improve program user interface, add new options and fix various issues across entire sitemap generator program including in sitemap builder, sitemaps ping etc.

For those of you creating XML sitemaps using A1 Sitemap Generator there are some excellent news.

I have been hard at work in engineering the website scanner engine with a special focus on improving the already fast crawler while also have it consume much less memory. I am happy to report I succeeded at both

Overall this update focused on:

• Increase crawler speed
• Minimize memory usage
• Various user interface improvements
• Various crawler engine improvements

If you are already a registered user of A1 Sitemap Generator, updating is as easy as downloading and installing the newest version.

Earlier today I wrote an article with a comparison of website sitemaps. The sitemap types explained include:

• Text file sitemaps.
• HTML and XHTML sitemaps.
• RSS sitemaps – RSS feed sitemaps are often used by blogs.
• ROR sitemaps – really RSS feed sitemaps with some custom extensions.
• XML sitemaps – also called Google Sitemaps and XML Sitemaps Protocol.

In my comparison of different site map kinds, I also explain:

• Background information and history
• Level of support by search engines
• Common and best usage tips and tricks
• Complete sitemap file examples

Have you ever wondered if ROR is actual RSS + some extensions? And howcome XHTML, ROR, RSS and XML sitemaps protocol all use XML, but still result in 3½ different sitemaps? Then read my article about website sitemap differences today!

I have recently written an article about crawling of websites which explains and shows how to solve many common crawling problems.

Things covered include:

• Having multiple domains with mirrored and/or distributed content.
• Consistent usage of www (or not) in front of domain names.
• Usage of Javascript for navigation.
• Broken links and redirects.

And many other things, about 10 in total, that can result in problems for a website crawler. The article focuses on solving all website crawl and scan problems using A1 Sitemap Generator. For those who have websites where crawler problems can not be solved, the article also shows how you can still create complete HTML and XML sitemaps for your website by configuring the sitemap creator program.

Read the article and make your website crawlable by search engines now!

In this post I am going to talk about the keyword explosion tools A1 Keyword Research has for growing an existing keyword list. A future post will address how it can research and build keyword lists nearly from scratch.

We are going to address following issues:

• Combine and tumble keyword lists
• Permutation of phrases in a keyword list
• How to cover typo and spelling errors
• Various tools to clean your keyword lists

How to combine and tumble keyword lists together:

Assume you have following 3 keyword lists:

Keyword list group #1:

• keywords
• keyword

Keyword list group #2:

• research
• marketing

Keyword list group #3:

• program
• software
• tool

Use tumble to input to get following keyword phrases:

• keyword marketing program
• keyword marketing software
• keyword marketing tool
• keyword research program
• keyword research software
• keyword research tool
• keywords marketing program
• keywords marketing software
• keywords marketing tool
• keywords research program
• keywords research software
• keywords research tool

Have permutation cover all word orders in all phrases:

Assume you have following keyword phrase in “input”:

• keyword marketing program

The word order tool can permutate above sentence into:

• keyword marketing program
• keyword program marketing
• marketing keyword program
• marketing program keyword
• program keyword marketing
• program marketing keyword

How to take your keyword lists to the next level:

Can you imagine the number of thousands of phrases above two tools can generate for you? Well, we are not done! Do you have customers that found you using spelling and typo errors in their searches? Probably not… But maybe you will soon. Move your keyword list into “input” and get ready to use following 3 tools:

• “Missing space”. Example: “keyword research” adds “keywordresearch”.
• “Missing letter”. Example: “keyword marketing” adds “keyword markeing”, “keyword markting” etc.
• “Switched letter”. Example: “keyword tool” adds e.g. “kewyord tool”, “keywrod tool” etc.

Cleaning and sorting your final keyword list:

Everyone is different, but if you like to have neat keywords lists here are the tools you need:

• Filter on min/max characters/words
• Filter on allow list of characters (to remove illegal)
• Lower case all keyword phrases
• Sort keyword phrases
• Trim for superfluous spaces
• Remove words repeated, e.g. “phrase phrase expansion”
• Remove phrase duplicates, e.g. “research program” and “research program”
• Remove permutations, e.g. “keywords explode” and “explode keywords”

Above is just one of many tools available in A1 Keyword Research. I will showcase more of them in this blog as time permits

Some time ago I created my first ever video about A1 Sitemap Generator and XML sitemaps as used by Google, Yahoo and other search engines.

However, one response I got was the question:
“You should tell us why we should pay you $39.00, when so many others are free!”

My initial response was “and so many of them are IMO crap”

I often get emails from people who have tried them all ending up very frustrated until they tried my tool.

Some unique features and benefits:

• For medicore sized websites, like 50.000 to 100.000 pages, my program can give much better linking and redirect information for all pages than other tools.
• Do you have a website large than above numbers? If you can live without detailed information like how all pages link with each other, you can change some scanning options and the limit becomes ~350.000.
• A1 Sitemap Generator can also calculate priority values for pages based on internal linking structure. Simply put, The more linked a page is, the higher its priority value.

Some website scanning options:

• Support for website crawler to “login” and scan password protected areas.
• Optionally obey robots.txt and nofollow/noidex attributes.
• Can go through proxies.
• Change useragent ID.
• Find links in CSS and javascript.
• Verify links to external pages still exist.
• Cutout session IDs.

Some sitemap creation options:

• Very rich HTML sitemap template system.
• Can automatically GZip XML sitemaps.
• Split sitemaps into multiple files when a limit is reached, e.g. 50.000 pages per XML sitemap.
• Create sitemap index file of generated XML sitemap files.

And those are just the simplest options. For those who need more control about website scanning and sitemap creation, they have it. There are also some qualities that are hard to explain, but still exist. For example how many types of links are correctly understood and normalized.

If I had to mention one more thing, it would be the sitemap/ignore/output filters one can configure the website scanner with. This, together with the automation support, makes is doable to maintain sitemaps for huge websites since one can very finely split scanning into sections. I believe I also here offer more options for the webmaster compared to the competition.

If you have suggestion as to what you would like more information about in future videos, please let me by posting a comment

After Google/Yahoo/MSN announced support for XML sitemaps protocol, it was only a matter of time before more would follow. Yesterday it was announced that Ask.com and IBM have been added to the list of XML sitemap protocol supporters.

One of the hottest news seem to be that it is now possible to link your xml sitemap from your robots.txt file.

You can simply add this line to your robots.txt file:
Sitemap: http://www.example.com/sitemap.xml

I think this change will help make the XML sitemaps protocol a widely used standard. The primary reason being it enables search engines to automatically discover XML sitemaps on their own.

For more information, you can check two recent blog posts by Google and Yahoo.