Lost Knowledge - The Issues of Digital Volatility

Ask yourself a few simple questions:

• Are you able to keep track of your digital data?
• Is your data safe from loss or damage?
• Can you find and access your data quickly and easily?
• Is your data still readable - even after some years?

Digital Volatility, by that I mean the permanency of digital data, is not just an issue for large companies, software developers, government organizations, accountants, doctors or lawyers. The issue extends to the everyday home or office computer user.

To begin, I would like to list a number of factors effecting the current “digital age”:

• Many households have a personal computer.
• High-capacity digital storage media is easily affordable and readily available.
• The majority of information or media is available as or can be converted to a digital format.
• Email users generally receive more emails per day than they do letters, faxes or in some cases, telephone calls.
• Internet-based platforms are commonly used for personal communication and networking.
• Text messaging services are commonly used for personal short message communication.

With these factors in mind - consider the following (simple) questions with regard to your digital assets:

• Would you be able to find and read an email or electronic document that you sent to someone four years ago quickly?
• If you could find that email or document, would you be able to find all of the other emails or documents that are related to it quickly?
• If your hard disk drive failed tomorrow, would it cause you to lose valuable data?
• If you were lucky and it did not cause data loss, how much time would it cost to recover the data?
• If you wanted to find any single music file that you have - could you do so quickly?
• If you needed a telephone number, address, email-address or similar contact details - would you require an electronic device to obtain it?
• If you lost your mobile phone, would you also lose important information?

These questions and moreover the answers that are likely to arise as a result of asking them, generally highlight a concerning issue. We have become active authors, collectors and users of digital information; however we are generally inefficient and slovenly when it comes to preserving it on a long-term basis. Unless we rectify this, we will almost certainly find that we will lose control of our information and probably, given time, lose our information altogether.

Let us identify some of the digital information that we leave around - both on our own storage media and with online services, in the hope of finding and using it again someday:

• Photos
• Documents (Letters, Faxes, Orders, Bills, Bookings, Tickets)
• Contact Information (address books, birthday lists, etc.)
• Calendars
• Electronic Mail (sent and received, along with attached files)
• Electronic Books
• Music
• Films
• Links to Websites
• Computer Software
• Work-Related Files
• Other Files

Now that we have started to consider the kind of digital information we accumulate, let us now take a brief look at the kind of obvious risks there are, that may affect our ability to use that digital information in a month, a year or even in ten years from now:

• Physical damage to storage media (e.g. accident, fire, flood, voltage spike, sabotage, etc.)
• Theft of storage media
• Misplacement of storage media
• Fatigue of storage media
• Corruption due to computer virus infection
• Corruption due to software errors
• Corruption due to hardware errors
• Corruption due to power failure
• Inaccessibility due to password-related issues (e.g. loss or expiration)
• Carelessness (e.g. accidental deletion, formatting)
• Unavailability of suitable software to use/read files (e.g. an old file format)
• Incompatibility between available software and software used to create the files
• Short-term unavailability due to computer failure
• Short-term unavailability due to power failure
• Lack of indexing and cross-referencing of information for all storage media
• Inefficient searching of data due to distribution amongst multiple storage media
• Unavailability of a web-based service due to own technical difficulties
• Unavailability of a web-based service due to service provider's technical difficulties
• Unavailability of a web-based service due to bankruptcy or closure of a service provider
• Unavailability of a web-based service due to a change of policy (e.g. payment required)
• Unavailability of a web-based service due to unpaid fees
• Unavailability of a web-based service due to sabotage
• Unavailability of a web-based service due to a lack of Internet connection (e.g. for security reasons, service disruption)
• Data theft from own computer or web-based service due to hacking

These are some of the most likely and most obvious reasons why it may prove difficult at some point in our lives, to get at our valuable data quickly and reliably and there are surely many more reasons waiting to prove their evil existence just lurking around the corner.

With these thoughts in mind - what can we do to improve our digital archival skills and to reduce the risk of data loss, improve the efficiency of data retrieval and get the most out of our own little digital libraries?

Storage media, especially portable hard disks, can often be thought of as storage boxes. Think about the way that you would pack and label boxes when you move house or store property away in a basement or attic and apply this knowledge to your storage media:

• Keep the number of storage media to a sensible minimum.
• Store similar data together on one storage media, rather than distributing it.
• Physically label storage media that you intend to use for a specific purpose, so that you can identify what is on it, without requiring a computer.
• Decide what is really important.
• Important files – files that if lost, would cause a real headache, should be kept as multiple copies, in different locations, ideally in a fire and water resistant container or safe.
• Handle storage media with care
• Avoid impacts (drops, bangs, etc.).
• Avoid high or low temperatures and direct sunlight.
• Avoid sources of magnetism (loud speakers, transformers, etc.).
• Avoid humid or wet environments (damp, rain, beverages, etc.).
• Check them on regular occasions to ensure that they are still intact.

Accompany important files with copies in alternative formats. This is only essential for files that are of genuine importance – where if they were to be lost, damaged or become unreadable, you would have a real problem.

• A text file can be read using even the simplest of means. Whilst formatting will be lost, the textual content of the document is available for quick reading, indexing, searching, etc.
• An HTML file can be read using many different programs and maintains a good deal of formatting - sufficient for reading a well-formatted document.
• A PDF file can be read using several different programs and maintains not only an attractive format, but is also suitable for producing printed versions.

• A comma-separated value (CSV) text file can be read using the simplest of means whilst maintaining a tabular structure.
• In addition, an HTML or PDF file is advisable in order to maintain formatted and printable equivalents of the spreadsheet.

• Avoid exotic or proprietary file formats. If you use a specific piece of software to create images, ensure that in addition to the software’s own file format, you also export those images into a more general-purpose format.
• PNG and JPEG formats are widely supported and are suitable for the majority of situations.
• Where possible, select a lossless compression or zero compression and keep the image at the original size for archival purposes - ensuring that the image remains as useable as possible, even for printing.

• Avoid exotic or proprietary file formats. If you use a specific piece of software to create audio files, ensure that in addition to the software’s own file format, you also export those audio files into a more general-purpose format.
• MP3 format is widely supported however it is, by definition a file format that stores audio in a compressed form that has also undergone some level of data loss.
• For self-made audio recordings, keep an MP3 and WAV equivalent file in addition to any files produced by the music/recording software.

• Keep a text-based equivalent of application-specific address book or calendar formats. The VCARD and iCalendar formats are text based and offer at least one text-based possibility.
• Programs such as Microsoft Outlook can export address books as Text, CSV, HTML, Excel or Access files – however Text and CSV are good long-term solutions here as their formats as are simple.

• Programs such as Microsoft Outlook can export emails as Text and HTML, furthermore it is often possible to export a CSV, Excel or Access file that contains the textual content of email folders.
• It is advisable to select the simplest format, such as Text or CSV.

• Compressed files such as ZIP, GZ, RAR, etc. are fine for the purpose for which they are intended – however they are not suitable for long-term archival storage.
• The file formats may at some point become obsolete.
• The data contained within them is more susceptible to corruption than if they were left uncompressed.

Like many aspects of safety, it never seems relevant – until it is too late. Words of regret and remorse, rarely help once the damage has already been done.

The nature of the digital world, is that things happen quickly – whilst the creation of data is becoming ever more simple, so (unfortunately) is the ability to cause massive damage and loss in the blink of an eye.

Be warned! Be aware! Be prepared!

Published on 14.09.2010 - Hedderley's Regular Impressions