The Enhanced Network Scanjet 5



Quick Links

If you are a first-time visitor, please do read this page first. If are already familiar with this project and are looking for something specific, here are some other pages on this site:

Updates

I have written a very brief guide on how to upgrade the installed Enhanced Network Scanjet 5 software release on a scanner that has already been "Enhanced" but with an older version. ( 2003-03-10 )

New Release! This release corrects a couple of small issues related to the document feeder: centering of the image on copies to a printer, and ability to do multi-part scans from multiple sources. The new release is available at the usual location. There is not yet a new installer CD image for this release. ( 2003-03-10 )

New translations for French (from Jerome Jourdain) and Swedish (from Magnus Bengtsson) are now available. Thanks to both. ( 2004-12-03 )

There is now an Italian translation file available, courtesy of Lorenzo Digiambattista. Thanks, Lorenzo! ( 2004-10-25 )

Installer CD available! I have created an installer CD for the Scanjet Upgrade code. This will install FreeBSD and all other needed software in one step and provides an easy way to bootstrap a scanner with a known working configuration that can be further configured and customized. Please see the installer web page for more information. ( 2004-10-16 )

There is a new version, 2004.10.15, that contains the network configuration control panel extension. This makes it possible to do basic network setup of the scanner from the keypad on the scanner itself. This way a new install can be bootstrapped onto the network so that ssh can be used to do the rest of the configuration. Several bugfixes were made to the network configuration tool as well, primarily to allow a read-only root filesystem. Also, the missing sjpwd.c file is included. Download here. ( 2004-10-15 )

There is now a Danish translation file available, courtesy of Kenneth Møller. Thanks, Kenneth! ( 2004-10-06 )

Now I've done some hardware upgrades to my scanner. Please see the Turbo Enhanced Network Scanjet 5 page for details. ( 2004-08-22 )

A year and two days since the last release. I've added a couple of minor features: an overall brightness setting in the configuration file, and the ability to configure an origin offset for each printer so that copies can be positioned on the page the same as the originals. I've also removed the "beta" designation, since I've received basically nobug reports during the past year. The code is in the usual place. ( 2004-08-16 )

Many months ago I wrote a configuration control panel extension to allow configuration of basic IP and other system parameters from the Scanjet control panel. The intention was to use this, along with a web-based configuration system, to allow adminitration of the scanner without the need for command line access, as a commercial add-on to the basic free package. I have not had time or interest in pursuing the commercial aspect further, so I have decided to release the control panel under the same BSD license as the rest of the software. It is available in the Files area. At some point I will incorporate this into the regular package. Note that this only works under FreeBSD (probably only later 4.x versions, at that) and I have no intention of adding support for other operating systems. If someone else wishes to do the work for Linux or whatever, I will be glad to incorporate it into the standard distribution. To use this, simply drop into /usr/local/bin. The hook for it is already in the standard code, just type 'CONF' and hit enter on the panel.( 2004-04-30 )

A German-language plug-in is now available, courtesy of Harun Basalamah. Harun points out that the translation might not be complete since he doesn't use all of the new features at his location. Thanks, Harun, for sharing this and also helping to find bugs in the new code! ( 2003-08-15 )

More beta fixes. Several problems in fax handling are now fixed in the 2003.08.14 beta, also some bash/Linux compatibility issues and a couple of problems with gzip compression. Keep testing! ( 2003-08-14 )

I have fixed a few things in the 2003.08.02 beta release mostly relating to the new multi-part scan mode, also a few other smaller problems. It's better, but probably still beta quality. ( 2003-08-10 )

Major Update! I am releasing a beta-quality version of the new scanner code. This is a major update with many new features, and frankly, I'm tired of testing it. I'm sure there are still bugs, but it generally works. Thanks to all who provided input for features. To get an idea of the new features, take a look at the default configuration and translation files. Please give this a try if you can and let me know of any problems you might have. I'm sure you will like the new features! ( 2003-08-02 )

I finally got around to looking into the situation with the Olicom OC-2173 10Mbit ethernet card that comes in the Scanjet. There are a few differences with a standard Intel EtherExpress card (which shares the same chip) but I seem to have worked them out. I now have a kernel patch for FreeBSD 4.3 through 4.8 that will make the card work with the "ex" driver that is available here. ( 2003-08-02 )

Local configuration setting may now be put in /usr/local/etc/sjrun.conf so that they won't get overwritten by reinstalls. ( 2003-07-24 )

Rather than individual files, I am now releasing code as distribution tar files, which also include some new documentation and a Makefile that automates building and installing the programs. There is also a new script, 'sjctl', included that can be used to start, stop, and restart the scanner software. The makefile will install this script both for manual use and also to start the scanner software automatically at bootup. ( 2003-07-21 )

Yet another front panel "Easter Egg". Typing "VERS" ("8377") and pressing Enter on the keypad will display the version of the distribution installed on the scanner. Also, the program previously named 'pbmtolj' is now called 'pbm2lj' to avoid conflict with the netpbm program of the same prior name. ( 2003-07-21 )

There is a new "To Do" list of things that I think should be added but haven't gotten around to yet. ( 2003-07-17 )

With the current sjrun.sh script, the scanner can now display it's IP address by typing "ADDR" ("2337") and pressing Enter on the keypad. Thanks to Julian Stacey for the inspiration for this feature. This is in addition to the previous (undocumented) features accessed by typing "BOOT" ("2668") or "HALT" ("4258"), which repectively reboot, and halt the operating system on the scanner. ( 2003-07-17 )

Ayis Pyrros has sucessfully upgraded his Scanjet using Linux (Redhat 9.0) rather than FreeBSD! The keyboard/display translater (sjlcd.c) ran fine as-is, but I needed to make some changes to the shell script to work correctly with the Bash shell provided with Linux. The new script, sjrun.sh, linked from this page, now works under both Linux and FreeBSD! My thanks to Ayis for his feedback and for testing my changes! ( 2003-07-17 )

I have fixed a couple of bugs in sjrun.sh pointed out by Julian Stacey. First is the inability to save a TIFF format scan. Second is a mishandling of fax resolution in fine mode. The version linked here now has the fixes. ( 2003-07-10 )

Julian Stacey is in the process of getting his Scanjet converted and has posted some information from his experience on his Scanjet 5 Conversion From NT to FreeBSD page. ( 2003-07-09 )

If you are new to FreeBSD or just want to take the easy way out, be sure to look into the FreeBSD's Ports collection. This is an easy way to install the needed Ghostscript, SANE, and libtiff (listed as 'tiff') components. There are even precompiled packages for these items that will download and install each of them with only one command! ( 2003-05-13 )

I have been asked for the modified Olicom token ring driver I refer to below. The version I use on my Scanjet is available here. This file replaces /usr/src/sys/contrib/dev/oltr/if_oltr.c in FreeBSD 4.7. It may very well work in other 4.x versions as well. This driver is only known to add support for the OC-3118 card in plug and play mode. I am working on a better version but don't have it ready. I don't know if this is what card HP used in the token ring Scanjets as mine was ethernet originally. ( 2003-05-09 )

Magnus Bengtsson has successfully "Enhanced" his Scanjet. Magnus opted to use FreeBSD 5.0 and has helped greatly in finding issues with using this version compared to 4.7, which the software was originally developed on. Thanks, Magnus, and good work! ( 2003-05-02 )

I have added a "Problems" section at the end of this page with a few common problems and their solutions. Please let me know if you find other things that should be listed here! ( 2003-04-30 )

I have made a few changes in sjlcd.c that make it work correctly on FreeBSD 5.0. Some termios defaults are different, so I needed to explicitly set a few more things. The copy linked on this page has the updates. ( 2003-04-29 )

A few people have contacted me with problems getting things running just right, and all are using FreeBSD 5.0. Unless you have some experience and are willing to work through some problems (I'll try to help!) I suggest you stick with FreeBSD 4.7 or 4.8. ( 2003-04-25 )

I have found a couple of small bugs in sjrun.sh since it was originally posted here. They are fixed now in the copy linked on this page. I'm sure there will be more. ( 2003-03-01 )

Background

I recently purchased an HP Network Scanjet 5 with the intention of using it principally as a copier (in conjunction with a laser printer) and a fax machine. At the time I purchased it (on eBay) I did not fully realize that the scanner was not a fully-functional, standalone unit. It turns out that it depends heavily on a service running on a Windows NT server (which I don't have on my network).

A quick search on Google revealed some interesting information: Project: Network Scanjet "Repair"

Long story made short, a couple of gentlemen in Germany found their Network Scanjet to be nearly worthless due to lack of recent software support from HP. Upon disassembling the unit, they found that it was based on PC-compatible hardware and were able to load Linux on it. The scanner internals are the same as a regular Scanjet 5, so the scanner is fully supported by SANE. Running SANE as a daemon on the Scanjet, they were able to restore functionaly by remotely controlling it from a SANE frontend elsewhere on the network.

This intrigued and inspired me. I determined to duplicate their work, with a few changes:

Hardware Upgrades

As I wanted to preserve the original HP software for study, I saved the original drive and instead used a Quantum 2.5GB unit I had lying around. It's actually a drive I got in a Motorola StarMax Macintosh clone I picked up a while back to experiment with.

The scanner originally had 4MB of memory, probably kind of tight, so I dug out an old 16MB 72-pin SIMM to replace the 2MB SIMM that came in the unit. This brought the the total memory to 18MB, plenty for what I wanted to do. Later I further upgraded to 34MB by replacement with a 32MB SIMM.

Lastly, I (eventually) replaced the 10Mb ethernet card that came in the scanner with a token-ring card. I use a token ring network for printing on my home network because token-ring print servers (and other hardware) are available very cheap as few want them anymore. Since I was intending to use the scanner mostly as a copier, it made sense to put it on the token ring network where the laser printer is.

Operating System

Rather than Linux, I used FreeBSD. I am much more familiar and comfortable with it, plus I have other FreeBSD machines running. This was important, as the processor in the scanner is a 486DX2/66 and I did not want to have to (slowly) compile the software packages I needed on the scanner itself.

Although the motherboard in the scanner has floppy disk controller circuitry, the connector is missing from the board, so booting from floppy was out unless I felt like soldering the connector in (I didn't), and the BIOS on the machine does not support booting from CD. So, I temporarily installed the drive in my desktop machine to load.

After loading the drive and installing it in the scanner, along with an old ISA video card, FreeBSD booted right up to the point of scanning the SCSI bus, and hung there, producing a timeout message every couple of minutes. Looked like a missing interrupt problem to me, and sure enough, a quick check of the helpful hardware notes on the German site showed the SCSI controller to be on IRQ 10, where FreeBSD detected it on IRQ 12. Apparently the Adaptec chip is not wired to the board 100% correctly. Hardcoding the IRQ config in the kernel fixed the problem and I was up and running:

Copyright (c) 1992-2002 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 4.7-RELEASE #4: Sun Feb 23 14:41:42 EST 2003
    david@madole.net:/usr/src/sys/compile/SCANJET
Timecounter "i8254"  frequency 1193182 Hz
CPU: AMD Am486DX2/4 Write-Through (486-class CPU)
  Origin = "AuthenticAMD"  Id = 0x434  Stepping = 4
  Features=0x1<FPU>
real memory  = 18874368 (18432K bytes)
avail memory = 15683584 (15316K bytes)
Preloaded elf kernel "kernel" at 0xc02e5000.
pcibios: No call entry point
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
isa0: <ISA bus> on motherboard
ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
aic0: <Adaptec 6260/6360 SCSI controller> at port 0x140-0x15f irq 10 on isa0
aic0: aic6360, dma, disconnection, parity check
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
sio0 at port 0x3f8-0x3ff irq 4 on isa0
sio0: type 16550A
oltr0: <Olicom Token-Ring ISA 16/4> at port 0xa00-0xa1f iomem 0xc0000-0xc1fff irq 3 drq 1 on isa0
oltr0: MAC address 00:00:83:b3:90:08
ad0: 2445MB <QUANTUM SIROCCO2550A> [4969/16/63] at ata0-master BIOSPIO
Waiting 3 seconds for SCSI devices to settle
pass0 at aic0 bus 0 target 1 lun 0
pass0: <HP C1301A 3610> Fixed Processor SCSI-2 device
pass0: 3.300MB/s transfers
Mounting root from ufs:/dev/ad0s1a

The Display

One of my key goals was to have the scanner work as a completely stand-alone device and be very easy to use. This made the use of the display and keypad on the scanner a necessity.

The display and keyboard connects through a couple of non-standard cables to a standard 10-pin header serial port on the motherboard. I put together an extension cable to go between the motherboard and the proprietary cables from which I tapped out the transmit data (TXD) and receive data (RXD) pins.

By connecting these pins to the receive data line of my desktop machine, I was able to watch the traffic beween the motherboard and the display unit while the original HP software was still loaded. Although I did not have the NT server component, the software booted and got far enough that I had plenty of example data to reverse-engineer.

It turns out that the display runs as 115200 baud, 8 bits, even parity, and two stop bits (important!). The display supports a text mode that the HP software uses to display a couple of status messages while booting, and also a graphics mode, which is used once the software is up. For my purposes, the text mode was plenty, although I have partially figured out the graphics mode as well, should I want to do something fancier in the future.

The data format to the display is far from straight ASCII. There is some kind of command structure that uses a '0xa7' character as an escape or sync byte. Although I don't know exactly what most of the datastream means, I figured out enough to make it usable, and I just send the rest without knowning what it means.

Input from the keypad uses a similar command structure, which I figured out enough to use as well. Lastly, there are three LED indicators on the front panel that I figured out how to light and flash on command.

The Software

I ended up creating four software components used to operate the scanner, which I am making available here. There are also brief descriptions of the other freely available software components I have relied on.

sjlcd.c

This is a "driver" of sorts for the LCD display and keypad. Rather than creating an actual device driver, which would have been more difficult, and less portable to other operating systems, I instead chose a different approach.

The sjlcd program places itself between the display hardware and a child process. It reads output from the child process and converts to a format suitable for the display, and reads from the keypad and converts to a format that is easy for the child process to use.

The output format from the child is ASCII text, with a few control characters and VT100 escape sequences recognized to allow clearing the display, positioning the cursor, and controlling the front-panel LCDs.

The input format is text strings corresponding the the labels on the keypad followed by a newline character. This allows line-oriented processing of input by the child process -- each keystroke is one line.

The program is run like this:

sjlcd /dev/cuaa0 sjrun

Where /dev/cuaa0 is the call-out device for the serial port the display is attached to. The program takes care of setting up the correct baud rate and other settings, and then runs the program named by the second argument, processing all it's input and output

This program is known to work on both FreeBSD and Linux, and will probably work on other Unix-like operating systems as well.

sjrun.sh

This is the actual interactive part of the scanner software that makes it work. Yes, it's a shell script!

This code allows the user to scan single- or double-sided documents and send the output to either a PCL printer, an email address, a fax number, or save it to the user's home directory on an NFS-mounted server. It includes a number of other options to adjust scanning brightness, set lineart or grayscale/halftone mode, print single- or double-sided copies, reduce or enlarge the size from the original, etc.

The user interface used is very simple and easy to operate.

This script runs under both the FreeBSD Bourne shell and Bash on Linux.

There are some settings in the beginning of the file that you will probably want to modify to suit your environment. Take a look before you run it.

pbm2lj.c

To use the scanner as a copier, I needed to get scanned documents printed on my IBM Network Printer 17, which accepts PCL input. I used the pbmtolj converter that is part of the Netpbm package to convert pbm-format scan files from SANE to PCL output.

The only problem was that on the 486 processor in the scanner, it was s-l-o-w, taking ten seconds per page to convert. Looking into the pbmtolj source, I could see why -- it was very inefficient in how it processed the file. By rewriting it, I was able to speed it up by better than a factor of ten, now converting a page in under a second. This was usable.

In the process of updating pbmtolj, I also removed all need for the Netpbm package. My version of pbmtolj -- pbm2lj -- can be compiled completely standalone without the Netpbm libraries. In fact, no part of Netpbm is used at all on my scanjet anymore.

At some point, I may rewrite this totally to work with TIFF input files through use of libtiff.

tiffres.c

To implement enlarging and reducing the original, I change the scan resolution that I ask from SANE. This works great and avoids the need to rescale the images after the scan. The only problem is that SANE records the resolution in the TIFF images it outputs and other tools, like tiff2ps, use that information correctly (incorrectly in this case) scale the image.

This small program modifies a TIFF file in place, replacing the resolution stored in the file with a new one. It's a real quick-and-dirty hack that doesn't rely on any external libraries or other code.

tiffrotate.c

To allow for different page orientations (landscape, upside-down, etc), I needed a way to rotate the image. Where Postscript output is being done this way easy enough to do by changing the coordinate system transform before doing the conversion.

For other cases, where the end result is a TIFF file, I needed a utility that could rotate TIFF file. I looked into existing solutions like ImageMagick, but they were all very slow. I ended up writing a program against libtiff that handles the cases needed and runs very quickly even on the scanner's slow processor. ATiffrotate.c can handle 1-bit and 8-bit images and rotates in 90-degree increments.

SANE

SANE (Scanner Access Now Easy) is used to drive the scanner component of the scanjet. The scanner is completely supported, including the automatic document feeder. The "hp" backend is used along with the "scanimage" command line tool. For printing, scans are output in pbm format, for faxing, email, and saved scans, TIFF is used.

The libtiff Utilities

From the libtiff utilites, I use the tiff2ps program to conver TIFF scan files to Postscript as a first part of creating PDF files. Incidentally, this program is much faster than the similar pnmtops that is included in the Netpbm library.

I also use the tiffcp program to compress TIFF scan files and produce a single, multi-page TIFF for faxing and emailing

Ghostscript

Ghostscript is used to produce PDF images by piping in the output of tiff2ps from the tiff utilities and outputing using the pdfwriter output driver.

I use GNU Ghostscript on my Scanjet, although I don't know of any reason that AFPL Ghostscript won't work just as well.

FreeBSD

This is one sweet operating system. Note that I am using version 4.7, not the latest 5.0. The 5.0 release is still considered experimental and 4.x is recommended for production use. The current "STABLE" version, 4.8, should be fine.

Here is the kernel config file I use on the scanner (downloadable here):

machine		i386
cpu		I486_CPU
ident		SCANJET
maxusers  	0

options   	INET
options   	FFS
options   	FFS_ROOT
options   	NFS
options   	PROCFS
options   	COMPAT_43
options   	SCSI_DELAY=3000

device		pci
device		isa

device		ata0	at isa? port IO_WD1 irq 14
device		atadisk

device		aic0	at isa? port 0x140 irq 10
device		scbus
device		pass

device		atkbdc0	at isa? port IO_KBD
device		atkbd0	at atkbdc? irq 1 flags 0x1
device		vga0	at isa?
device		sc0	at isa? flags 0x100

device		npx0	at nexus? port IO_NPX irq 13

device		sio0	at isa? port IO_COM1 irq 4

device		oltr

pseudo-device	loop
pseudo-device	ether
pseudo-device	token
pseudo-device	pty
pseudo-device	bpf

The token-ring driver is not stock FreeBSD, as the oltr driver does not support ISA cards. I added support myself for the Olicom OC-3118 and a couple other cards. As soon as I get it cleaned up, I will be submitting the updated driver for inclusion in FreeBSD.

Unless you happen to be using an Olicom token ring card as well, you will need to replace the "device oltr" configuration line with the appropriate driver for whatever network card you are using.

I have since made some minor changes that also allow this software to work under Linux. Since I don't actually have Linux running on my own scanner, I can't provide more specific details on operting system setup or configuration for Linux.

Downloading

Source distributions are available through HTTP or FTP. Please see the README and INSTALL files within the distribution for more information.

Summary

With the addition of this new software, the scanner has many more features than with the original HP software. As an added bonus, it is a completely stand-alone device requiring no other server to function.

As written, the software supports copying documents to a PCL printer, although changing this to support a Postscript printer would be trivial using the same tiff2ps utility that is used to produce PDF files.

It also supports sending faxes using the MaxEmail email-to-fax service. It would be trivial to change it to use the EFax service, although why would anyone want to? Having used both, I can say without any reservation that MaxEmail is far superior, both in operation and customer service.

It would even be possible to fit a short modem card into the extra ISA slot inside the scanner and use something like HylaFAX to implement direct faxing through a phone line.

The scanner will also produce PDF or TIFF files of scanned pages and email them to any user in /etc/passwd or in a particular group (or the NIS versions of the same). Or, it can save the scan to a subdirectory of the user's home directory via an NFS mount.

If you have any questions on any of the software I developed, or would like additional information, please feel free to contact me at david@madole.net.

I would enjoy hearing from you if you implement any of this software on your own Scanjet, especially if you make any improvements!

Screenshots

What software web page would be complete without screenshots?

Here is the main menu displayed when the scanner is idle:

The four soft keys under the display select the operating mode.

Pressing the first soft key places the scanner into copy mode. This prompts for the number of copies in the main display area, and displays options above each of the soft keys, which can be used to change those options:

The copy count can be changed with the up and down arrow keys, of by directly keying in a number on the keypad.

Pressing any of the soft keys changes the setting displayed above it. If while at the above display, the second and fourth soft keys were pressed, then the third, the display would end up here:

When the second key was press, the output copies changed from single-sided to double-sided, when the fourth was pressed, the scan mode changed from black and while to grays and color (which corresponds to the "lineart" and "halftone" settings in SANE in this case). Then when the third key was pressed, the main display area prompted for the change in brightness, again, the arrow keys or the keypad may be used to change this.

The diplay is similar for other operating modes. If "Email" is chosen from the main menu, the display prompts for what user to email to:

Since single- or double-sided output is meaningless for email, the second soft key instead changes the document format between PDF and TIFF.

The first soft key always selects "Resize", allowing the output page to be scaled between 20% and 500%:

This shot is also from fax mode, showing that the second soft key here selects between normal and fine detail transmission mode

At any time, the "Start" key (the large green one, see the overall panel picture at the top) can be pressed to start operation. The red Stop key can be pressed at any time to cancel any operation in progress and return to the top menu.

If left unattended for 30 seconds in any mode, the scanner automatically resets all settings and returns to the top menu.

Problems

Problem: FreeBSD freezes up while scanning the SCSI bus during boot.

Solution: Be sure to recompile the kernel with changed settings for the aic device. The SCSI controller on the Scanjet motherboard has it's IRQ miswired and gets auto-detected as IRQ 12 when it is really on IRQ 10. Hard coding this in the kernel configuration prevents auto-detection.

Problem: The original ethernet card is not recognized by FreeBSD.

Solution: Use a different card that is supported. The Olivetti ethernet card that comes in the scanner is based on an Intel chipset that is supported, but the plug-and-play IDs are not in the driver and the code to set the MAC address doesn't work right, either. I may fix the driver when I have a chance, but I don't use the card myself, so it might make me a while!

Problem: Nothing comes up on the display when sjlcd is run.

Solution: Make sure that you recompile the kernel and do not have "flags 0x10" on the sio0 device. This setting enables use of the serial port as a console and seems to mess up the baud rate setting (at least on FreeBSD 5.0).

Problem: Difficulty compiling Ghostscript or SANE from source.

Solution: These programs require GNU 'make' to compile, not the BSD 'make' that that installs with FreeBSD! I like to install GNU make as 'gmake' to avoid confusion over which one is being used. This can be done easily by using the '--program-prefix=g' option to 'configure' when building GNU 'make'.

To Do

If a standard FreeBSD port and package were created for my code, it would make installation easier. Proper dependencies could be put in for Ghostscript, libtiff, etc., so that installation could be done with just one command. Likewise, on the Linux side, someone could make an RPM.

References

Project: Network Scanjet "Repair", maintained by Matthias Meixner and Christian Haul: The inspiration for my work, has some good information on the Scanjet motherboard hardware including some photos of components and links to display controller information. The work done here is based on Linux.

Scanjet 5 Conversion From NT to FreeBSD, maintained by Julian Stacey: Contains supplemental files and notes based on on my work. Julian has also offerred to host a Scanjet conversion mailing list here.



Web hosting by
Omd3.com Hosting Services
David S. Madole
2004-12-03 1415 GMT