Troubleshooting a non-connecting Bruker ELEXSYS spectrometer

Recently in the lab we've had some problems thanks to a helpful software update rendering it impossible to connect to a spectrometer. This troubleshooting provides a record of my steps to diagnosing the problem. Hopefully shortening the process in the future. The spectrometer in question is a Bruker ELEXSYS E580/E680 hybrid connected to a HP/Bruker desktop running openSUSE 12.1 x64, with Xepr v2.6b53. You will need super user privileges for almost all of these commands.

If not installed getminicom

su -

Enter root password.

minicom

Make sure that the RS232cable is connected from the spectrometers RS1 port of the ACPU board (bottom left) to the PCs RS232 port. By default the serial port should be/dev/ttyS0​if minicom is listening to anything else then change it. Do this by pressing Ctrl+A, O. Select A and change the device to the correct serial port. If you change the port exit minicom and reload it.

Otherwise turn on the spectrometers console (be careful with cooling water supplies, you dont want to burn out your microwave source), and minicomshould jump into life and read out the console's boot process

4X: SELF-TEST Rev. 1.8

Duart1 test: completed

Mpu test: MC68040 completed

Clock test: completed

Mmu test: completed

Calender test: completed

Centronics test: not ready

Fpu test: completed

Ethernet test: completed

Flash Bank0 : No Flash

Flash Bank1 : No Flash

Checking Memory: 0016 Mbytes RAM detected

 

OS-9/68K System Bootstrap

 

 

Will try to Boot from network

To get the boot menu press Esc

 

Now trying to Boot from network.

inetboot: using P4x (MC68360) Ethernet @ 0xFFD82C00 MTU=1500

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

bootp: retry 1 with new timeout=5 seconds

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

Retry after 5 rewait

Retry 2 with new timeout=0

bootp: retry 2 with new timeout=5 seconds

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

bootp: retry 3 with new timeout=5 seconds

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

Retry after 5 rewait

Retry 4 with new timeout=30

bootp: exiting after 5 retry

This error occurred during the Internet boot boot driver!

The OS-9 error code is: #007:025.

 

Now trying to Reconfigure the boot system.

 

BOOT SYSTEM RECONFIGURATION

 

 

Boot drivers available:

 

1 - Boot from Teac SCSI floppy drive

2 - Boot from OMTI 5400 SCSI floppy drive

3 - Boot from SCSI hard drive

4 - Boot from SCSI tape drive

5 - Boot from ROM

6 - Boot from network

7 - Boot from VMEbus

 

The priority of these boot drivers can be set below and will remain

set even when power to your system is turned off. This priority

determines the order that the boot drivers will be selected when your

system is configured to boot automatically. It also determines the

order that they appear in the "boot driver menu" as well.

 

The current system boot driver priority is: 6

 

Enter the numbers of the boot drivers in the order

that you want them prioritized. ->

​Whereas it should read something like:

inetboot: using on-board MB86960 (NICE) MTU=1514

bootp: 0:20:15:1:0:13 broadcasting for server

bootp: recv waiting 50 ticks

GOT BOOTP RESPONSE from server 128.141.250.202!

My IP address will be: 128.141.227.182

My tftp bootfile is: /dd/TFTPBOOT/os9boot

My bootfile size is: 528 (512-byte) blocks

My subnet mask is: 255.255.255.0

<<no timeoffset tag>>

tftp: fetch 528 blocks

tftp: got UDP src=808dcd3d:138 dst=808dffff:138 rewaiting

So there is clearly a bootp(boot-strap) or tftp(trivial file transfer protocol) problem.

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

src=00000000 means that a packet is being sent from a source with no IP address, which is what you'd expect from a spectrometer with no connection when it first boots. Whilst dst=ffffffff means that the packet is being sent to a broadcast address - ie anyone.

Check /etc/services and see if there are UDP entries for the bootstrapports. It should look like:

bootps 67/tcp # Bootstrap Protocol Server

bootps 67/udp # Bootstrap Protocol Server

bootpc 68/tcp # Bootstrap Protocol Client

bootpc 68/udp # Bootstrap Protocol Client

If bootp has been properly installed, note that DHCP uses the same ports.

Now see if the computer can "see" the packets from the spectrometer, in a terminal type

tcpdump -G 15 -i eth1 (or whatever the spectrometer port is)

This will listen to the port for 15 seconds and report any incoming packets

tcpdump: WARNING: eth1: no IPv4 address assigned

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes

07:18:33.126606 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:15:17:48:04:44 (oui Unknown), length 300

07:18:36.129775 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:15:17:48:04:44 (oui Unknown), length 300

07:18:42.135842 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:15:17:48:04:44 (oui Unknown), length 300

This tells us:

1) That no IP address has been assigned

2) That the computer is receiving a request every few seconds

Therefore the problem must be with bootstrap assigning the IP address of the spectrometer.

Let's go take a look at that then

nano /etc/bootptab

.allOS9P4X :sm=255.255.255.0:td=/:\

:hd=/usr/xepr/AcquisitionServer/OS9/CPU/P4X/BOOTP:\

:bs=auto:hn:bf=os9boot:vm=rfc1084:

.allOS9P4X16MB :sm=255.255.255.0:td=/:\

:hd=/usr/xepr/AcquisitionServer/OS9/CPU/P4X/BOOTP_16MB:\

:bs=auto:hn:bf=os9boot:vm=rfc1084:

This gives the OS9 CPU boot locations. These look fine. bf=os9boot causes bootstrap to lookup bootfile.host and find the correct address

At the bottom of the file we have:

#

# individual hosts

#

E580:tc=.allOS9P4X16MB:ht=ethernet:ha=0x00409800008d:ip=192.168.1.10:

Patternjet:tc=.allTNTPATJET:ht=ethernet:ha=0x0000ad022512:ip=192.168.1.11:

Specjet:tc=.allTNTTSA:ht=ethernet:ha=0x0000ad030512:ip=192.168.1.12:

ifconfig eth1 down

ifconfig eth1 up

ifconfig -a

eth1 Link encap:Ethernet HWaddr 00:15:17:48:04:44

inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0

inet6 addr: fe80::215:17ff:fe48:444/64 Scope:Link

UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

RX packets:179 errors:0 dropped:0 overruns:0 frame:0

TX packets:521 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:61370 (59.9 Kb) TX bytes:166712 (162.8 Kb)

Interrupt:17 Memory:f0000000-f0020000

eth1 has clearly now got a network connection!

Rerunning the tcpdump command does not give anything, showing no packages, which is what you'd expect from a connected spectrometer not desperately seeking an IP address.

Now let's try Xepr.

Acquisition > Connect to spectrometer

Nothing. Time out error. Restart the spectrometer and ensure thatminicomis still running in a terminal window

P4X: SELF-TEST Rev. 1.8

Duart1 test: completed

Mpu test: MC68040 completed

Clock test: completed

Mmu test: completed

Calender test: completed

Centronics test: not ready

Fpu test: completed

Ethernet test: completed

Flash Bank0 : No Flash

Flash Bank1 : No Flash

Checking Memory: 0016 Mbytes RAM detected

 

OS-9/68K System Bootstrap

 

 

Will try to Boot from network

To get the boot menu press Esc

 

Now trying to Boot from network.

inetboot: using P4x (MC68360) Ethernet @ 0xFFD82C00 MTU=1500

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: retry 1 with new timeout=5 seconds

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: got UDP src=00000000:68 dst=ffffffff:67 rewaiting

bootp: retry 2 with new timeout=5 seconds

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: retry 3 with new timeout=5 seconds

bootp: 0:40:98:0:10:57 broadcasting for server

bootp: exiting after 5 retry

This error occurred during the Internet boot boot driver!

The OS-9 error code is: #007:025.

No joy. The console can still not find the network.

Try reinstalling Xepr server now that the bootstrap server is correctly installed.

This can be done with the command:

./usr/xepr/AcquisitionServer/admin/installServer

But make sure the server settings are correct first.

------------------------------------------------------------

| Acquisition Server Installation |

| |

| V2.6 2002/07/15 HAB |

------------------------------------------------------------

 

Installing files for spectrometer communication:

 

Removed lines from /etc/hosts

Appended lines to /tmp/hosts.frag

Appended lines to /etc/hosts

 

Installing files for spectrometer boot:

SuSE Linux release: 12.1, using xinetd

 

service bootps: using template file 'bootps' ...

Installed file /etc/xinetd.d/bootps

 

service tftp: using template file 'tftp-SuSE-9.3' ...

Installed file /etc/xinetd.d/tftp

Appended lines to /tmp/bootptab

Appended lines to /tmp/bootptab

Appended lines to /tmp/bootptab

Installed file /etc/bootptab

Removed lines from /etc/exports

Appended lines to /etc/exports

 

Installing spectrometers:

 

Installing 'E580': updated, now processing spectrometer configuration file ...

Removed lines from /usr/xepr/AcquisitionServer/OS9/Servers/E580/Spectrometer.ini

Removed lines from /usr/xepr/AcquisitionServer/OS9/Servers/E580/Spectrometer.ini

Removed lines from /usr/xepr/AcquisitionServer/OS9/Servers/E580/Spectrometer.ini

Removed lines from /usr/xepr/AcquisitionServer/OS9/Servers/E580/Spectrometer.ini

Appended lines to /usr/xepr/AcquisitionServer/OS9/Servers/E580/Spectrometer.ini

 

Installing utilities for editing native OS-9 format files:

 

Installed file /usr/bin/os9edit

Installed file /usr/bin/os9view

 

Restarting system services:

 

redirecting to systemctl

redirecting to systemctl

redirecting to systemctl

redirecting to systemctl

 

Acquisition Server installation completed.

Let's check the files changed by the installer to make sure that the installation was a success.

nano /etc/hosts

Has a lot of repeat entries in it from multiple attempts at installing the server, this was cleaned this up to:

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

127.0.0.2 epr-e680.domain.example.edu epr-e680

192.168.1.1 XEPR

192.168.1.10 E580

#

# Addresses of secondary network interface and spectrometers

#

192.168.1.1 XEPR

192.168.1.10 E580/etc/xinetd.d/tftp

The other files/etc/xinetd.d/bootps/etc/xinetd.d/tftp and/etc/exports all look as you would expect. Check/usr/xepr/AcquisitionServer/OS9/Servers/E580/Spectrometer.ini to check that the settings have been copied across.

Reboot the PC. Openminicomagain and turn on the spectrometer.

4X: SELF-TEST Rev. 1.8

Duart1 test: completed

Mpu test: MC68040 completed

Clock test: completed

Mmu test: completed

Calender test: completed

Centronics test: not ready

Fpu test: completed

Ethernet test: completed

Flash Bank0 : No Flash

Flash Bank1 : No Flash

Checking Memory: 0016 Mbytes RAM detected

 

OS-9/68K System Bootstrap

 

 

Will try to Boot from network

To get the boot menu press Esc

 

Now trying to Boot from network.

inetboot: using P4x (MC68360) Ethernet @ 0xFFD82C00 MTU=1500

bootp: 0:40:98:0:10:57 broadcasting for server

GOT BOOTP RESPONSE from server 192.168.1.1!

(epr-e680).

My IP address will be: 192.168.1.10

My tftp bootfile is: /usr/xepr/AcquisitionServer/OS9/CPU/P4X/BOOTP_16MB/os9boot

My bootfile size is: 1396 (512-byte) blocks

My subnet mask is: 255.255.255.0

<<no timeoffset tag>>

tftp: fetch 0 blocks

tftp: server response 192.168.1.1 port 37493

tftp: expected 0: received 1395:400 blocks (698Kb)

A valid OS-9 bootfile was found.

---------------------------------------------------------------------------

CPU Board Name: P4X

---------------------------------------------------------------------------

Starting ISP subsystem ...

+4

 

IP Routing Table:

 

 

Add gateway = 0.0.0.0 gateway = 192.168.1.1 to NET routing list

+8

---------------------------------------------------------------------------

Mounting '/xacq' (-m -r=1024 -w=1024) ... done

Mounting '/dd' (-m -r=1024 -w=1024) ... done

Mounting '/r0' (-m -r=1024 -w=1024) ... done

---------------------------------------------------------------------------

Mount Status 12:47:32

OS-9 Remote Gid/Uid Mount

Device System Map Point

------ ------ -------- -----

/xacq BootWS yes /usr/xepr/AcquisitionServer/OS9

/dd BootWS yes /usr/xepr/AcquisitionServer/OS9/CPU/P4X

/r0 BootWS yes /usr/xepr/AcquisitionServer/OS9/Servers/E580/r0

---------------------------------------------------------------------------

PROMPT=E580#

PATH=/dd/CMDS

HOME=/r0

AcqSvCPURootPath=/dd

AcqSvRootPath=/xacq/Servers

AcqSvCPUName=P4X

AcqSvHostName=E580

AcqSvGExecDir = /xacq/Servers/CMDS

AcqSvDataDirName = /r0

PROMPT=E580#

PATH=/dd/CMDS

HOME=/r0

AcqSvCPURootPath=/dd

AcqSvRootPath=/xacq/Servers

AcqSvCPUName=P4X

AcqSvHostName=E580

+22

 

AcqSvG: 'AcqServerD' started ...

---------------------------------------------------------------------------

E580#

This looks like a successful console boot. Open Xepr and try to connect to the spectrometer. Success!