Thursday, February 04, 2010

Coming soon - download old indexes from our website

In a blog post two weeks ago, I reviewed our efforts to make our pre-1976 grantor & grantee indexes available to registry users via the internet. Since then, with the help of the Secretary of State’s IT Department, we’ve made a breakthrough that will soon allow anyone with an internet connection full access to all of our indexes going back to 1620, (all done with existing resources and personnel).

First, some background: The searchable index that’s integrated into our computer system both here at the registry and on our website reaches back to 1976. For all the years prior to that, the index is maintained in a series of “electronic books” that are stored in PDF format. These “books” are organized by date range and letter of the alphabet. Many of them are also very large files. These files have been accessible at our computers in the registry for several years. Last year, when inexpensive flash drive storage devices reached 16GB storage capacity, we offered anyone who brought in a blank flash drive a full copy of the index at no charge. That’s where we stand now.

Here is how the new system will work: We have gained access to an FTP site that will allow anyone with an internet connection to login and download whatever index files that person might need, all free of charge. Customers will have to enter a username and password to gain access to this site, but we will post both on our own website for ease of use. Once you’ve logged in, transferring files from the FTP site (FTP stands for “File Transfer Protocol”) to your computer is as easy as copying the files from one folder to another. Unfortunately, the transfer rate is relatively slow: a 5MB file downloads in a little over a minute, but a 100MB file takes closer to 20 minutes to download.

Anyone who is a “heavy duty” user of the registry should still obtain a full copy of the index on their own thumb drive. This will allow you to copy the entire index to computer. This new service will most benefit those who need the old indexes only occasionally. Up until now, if these people were doing research at night or on the weekend, having to look in an old index has presented a substantial roadblock, at least until the registry opens for business. Once this new system is in place, however, such a user would be able to download the necessary index files 24/7 and press on with the search. I expect this new service will be fully operational by the middle of next week.

4 comments:

bostonbubble said...

"we’ve made a breakthrough that will soon allow anyone with an internet connection full access to all of our indexes going back to 1620"

Will it be possible to download indexes after 1976 too?

Dick said...

No. Because the post-1976 indexes are all freely available online, we have never converted them to static image pages which is what we've done with the pre-1976 indexes. Our preference would be to have the older indexes in a searchable database, but that would be such a long and expensive project that the scanned index books serve as a nice bridge to that future objective.

bostonbubble said...

"Because the post-1976 indexes are all freely available online, we have never converted them to static image pages which is what we've done with the pre-1976 indexes."

Thanks for the fast response. I actually didn't have static images in mind for the post-1976 indexes and was curious about the SQL database itself. Access to it is freely available via your search interface, but that just provides a single record at a time, which is slow and inefficient for compiling aggregate data. I was curious, is it possible to get everything post-1976 in whatever format you have already (so no conversion necessary)? Is that an option for the "thumb drive" users? Thanks for your help.

Dick said...

There is not an automated method for users to easily gather data for analytical use. Because I have access to the database itself, I have a limited ability to extract large amounts of data in a CSV format in response to customized queries. If there's something specific that interests you, send an email to lowelldeeds@comcast.net and we can discuss the possibilities off-line