Although the Internet has given us a worldwide infrastructure on which to build the universal library, much of the world knowledge, history, and literature is still trapped on paper in the basements of the world's traditional libraries. Many libraries and content owners are in the process of digitizing their collections. While many such efforts involve the painstaking process of converting paper documents to computer-friendly form, such as SGML based formats, the high cost of such conversions limits their extent. Scanning documents, and distributing the resulting images electronically is not only considerably cheaper, but also more faithful to the original document because it preserves its visual aspect.
Despite the quickly improving speed of network connections and computers, the number of scanned document images accessible on the Web today is relatively small. There are several reasons for this.
The first reason is the relatively high cost of scanning anything else but unbound sheets in black and white. This problem is slowly going away with the appearance of fast and low-cost color scanners with sheet feeders.
The second reason is that long-established image compression standards and file formats have proved inadequate for distributing scanned documents at high resolution, particularly color documents. Not only are the file sizes and download times impractical, the decoding and rendering times are also prohibitive. A typical magazine page scanned in color at 100 dpi in JPEG would typically occupy 100 KB to 200 KB , but the text would be hardly readable: insufficient for screen viewing and totally unacceptable for printing. The same page at 300 dpi would have sufficient quality for viewing and printing, but the file size would be 300 KB to 1000 KB at best, which is impractical for remote access. Another major problem is that a fully decoded 300 dpi color images of a letter-size page occupies 24 MB of memory and easily causes disk swapping.
The third reason is that digital documents are more than just a collection of individual page images. Pages in a scanned documents have a natural serial order. Special provision must be made to ensure that flipping pages be instantaneous and effortless so as to maintain a good user experience. Even more important, most existing document formats force users to download the entire document first before displaying a chosen page. However, users often want to jump to individual pages of the document without waiting for the entire document to download. Efficient browsing requires efficient random page access, fast sequential page flipping, and quick rendering. This can be achieved with a combination of advanced compression, pre-fetching, pre-decoding, caching, and progressive rendering. DjVu decomposes each page into multiple components (text, backgrounds, images, libraries of common shapes...) that may be shared by several pages and downloaded on demand. All these requirements call for a very sophisticated but parsimonious control mechanism to handle on-demand downloading, pre-fetching, decoding, caching, and progressive rendering of the page images. What is being considered here is not just a document image compression technique, but a whole platform for document delivery.
DjVu is an image compression technique, a document format, and a software platform for delivering documents images over the Internet that fulfills the above requirements.
The DjVu image compression is based on three technologies:
The DjVu technology is designed from the ground up to support the efficient delivery of digital documents over the Internet. It provides various ways to deal with multi-page documents, and various ways to enrich the content with hyper-links, meta-data, searchable text, etc.
When you type the URL of a multi-page document, the DjVu browser plugin starts downloading the whole file, but displays the first page as soon as it is available. You can immediately navigate to other pages using the DjVu toolbar. Suppose however that the document is stored on a remote web server. You can easily access the first page and see that this is not the document you wanted. Although you will never display the other pages the browser is transferring data for these pages and is wasting the bandwidth of your server (and the bandwidth of the Internet too). You could also see the summary of the document on the first page and jump to page 100. But page 100 cannot be displayed until data for pages 1 to 99 has been received. You may have to wait for the transmission of unnecessary page data. This second problem (the unnecessary wait) can be solved using the ``byte serving'' options of the HTTP/1.1 protocol. This option has to be supported by the web server, the proxies, the caches and the browser. Byte serving however does not solve the first problem (the waste of bandwidth).
The DjVu technology was initially created by a few researchers in AT&T Labs between 1995 and 1999. Lizardtech, Inc. ( ) then obtained a commercial license from AT&T and continued the development. They have now a variety of solutions for producing and distributing documents using the DjVu technology.
The DjVuZone web site ( ) is managed by the few AT&T Labs researchers who created the DjVu technology in the first place. We promote the DjVu technology by providing an independent source of information about DjVu.
Understanding how little room there is for a proprietary document format, Lizardtech released the DjVu Reference Library under the GNU Public License in December 2000. This library entirely defines the compression format and the elementary codecs. Six month later, Lizardtech released an updated DjVu Reference Library as well as the source code of the Unix viewer.
These two releases form the basis of our initial DjVuLibre software. We modified the build system to comply with the expectations of the open source community. Various bugs and portability issues have been fixed. We also tried to make it simpler to use and install, while preserving the essential structure of the Lizardtech releases.
The DjVuLibre software contains the following components:
DjVuLibre comes with a variety of specialized encoders, c44(1) for photographic images, cjb2(1) for bitonal images, and cpaldjvu(1) for images with few distinct colors. Although these encoders perform well in their specialized domain, they cannot handle complex tasks involving segmentation and multipage encoding.
The Lizardtech commercial products (see ). provide a way to perform these complex encoding tasks
Another solution is provided by the compression server at (). This machine uses pre-lizardtech prototype encoders from AT&T Labs and performs almost as well as the commercial Lizardtech encoders. Please note that the Any2DjVu compression server is compose comes with no guarantee, that nothing is done to ensure that your documents will remain confidential, and that there is only one computer working for the whole planet.
Numerous people have contributed to the DjVu source code during the last five years. Please submit a sourceforge bug report to update the following list.