[NMLUG] another boot issue

[ebrown at lanl.gov (Ed Brown)]

On Thu, 2004-04-01 at 11:00, michaelm@nnmcc.edu wrote:
> Ed Brown said:
> > Situation:
> > two servers, with:
> > motherboard with two onboard SCSI controllers
> > one internal SCSI drive, with linux installed
> >
> > and:
> > one external raid array, with just data, sometimes plugged into one
> > server (via a scsi connector on the back panel), sometimes plugged into
> > the other (The servers are brought down before before moving the raid
> > array around.)
> >
> > problem:
> > When the external array is plugged into the back panel connector, the
> > BIOS finds it first, and Linux also wants to identify it as /dev/sda
> > The mount points in /etc/fstab use 'LABELS', so they work however the
> > internal drive is recognized.  But the system won't boot at all when the
> > external array if first plugged in, unless you go into the regular BIOS
> > and select the second drive for booting.
> 
> Do you mean that the BIOS is looking for a boot loader on a drive in the array?

The array looks like a single drive to the on-board scsi controller. 
Yes, exactly, the BIOS tries to to find a boot loader there.

> 
> > ('hard drive' is only listed
> > once in the boot order selection; a separate menu item lets you choose
> > the order of discovered drives - this is refreshed each time the machine
> > boots, it isn't static)
> >
> > questions:
> > What mobo manufacturer madness would allow the external scsi connector
> > to have precedence in the discovery sequence?
> 
> Is it the BIOS or is it the assignment of the SCSI IDs?  SCSI will boot from
> drives with ID 0 or 1, and if one of these is assigned to the external port,
> could that be (part of) the problem?

Actually they are on different 'channels', scsi0 and scsi1,
corresponding to the two controllers.  Both have id 0:
(and there doesn't seem to be flexibility in the case for where things
plug into, it's all very hard-wired/built-in)

-----------
# dmesg |egrep 'SCSI|sd'
SCSI subsystem driver Revision: 1.00
scsi0 : Adaptec AIC79XX PCI-X SCSI HBA DRIVER, Rev 1.3.10
        <Adaptec AIC7902 Ultra320 SCSI adapter>
        aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz,
512 SCBs
scsi1 : Adaptec AIC79XX PCI-X SCSI HBA DRIVER, Rev 1.3.10
        <Adaptec AIC7902 Ultra320 SCSI adapter>
        aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 67-100Mhz,
512 SCBs
  Type:   Direct-Access                      ANSI SCSI revision: 03
  Type:   Direct-Access                      ANSI SCSI revision: 03
  Type:   Processor                          ANSI SCSI revision: 02
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi disk sdb at scsi1, channel 0, id 0, lun 0
SCSI device sda: 234438656 512-byte hdwr sectors (120033 MB)
 sda: sda1 < sda5 sda6 sda7 sda8 sda9 sda10 sda11 sda12 sda13 sda14
sda15 >
SCSI device sdb: 71687372 512-byte hdwr sectors (36704 MB)
 sdb: sdb1 sdb2 sdb3 sdb4 < sdb5 sdb6 sdb7 sdb8 >
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,23), internal journal
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,17), internal journal
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,22), internal journal
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,19), internal journal
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,18), internal journal
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,24), internal journal
-------------

> 
> > Why does linux make the same mistake?
> 
> Same reason?
> 
> > Any suggestions on how to overcome?
> 
> If nothing else, could you put a boot loader in the 'wrong' MBR that the
> BIOS insists on using that simply chains to the 'real' boot loader?

Yesterday I tried this, more or less.  With the system booted and
running, the raid array plugged in, I ran:
grub-install --root-directory=/boot /dev/sda

The root-dir switch was recommended in the info page for a system with a
separate /boot partition, but it had the effect of installing the second
stage grub stuff into /boot/boot/grub, not exactly what I would expect
(but possibly a blessing, in that I had all the original stuff in
/boot/grub to fall back on if things went awry).

Anyway, I did indeed get a grub prompt booting from the raid array
'disk', instead of just a flashing cursor, but no menu of course, since
that method of installing grub doesn't create a menu for you (anaconda
or whatever install program you use is supposed to do that)  Entering
the root, kernel and initrd info at the grub prompt almost got me going,
but I ended up with a kernel panic/initrd/pivotroot failure.  Didn't
have time to keep trying last night.  Suspect the /boot/boot/grub thing
might be a factor.  

Also, I'm not sure this track is going to give the results I want
anyway, which is for the system to boot whether or not the external disk
is plugged in, without having to fiddle with anything.  Do you think
that is doable?

Thanks Michael!
-Ed

> 
> > I think this is a hardware problem only, pre-grub, because after the
> > POST and BIOS sequences, we just see a flashing cursor, no grub or
> > kernel messages at all.
> >
> > thanks,
> > Ed
> >