Installing Debian 3.0 on servers with the Smart Array 64x controller

The installation of Debian Linux 3.0 Woody (I've tried 3.0r2) on HP/Compaq Proliants ML350 with the SmartArray 641/642 controller will fail because this controller is not recognized before kernel version 2.4.21 (3.0r0 uses 2.4.18 when installed with the bf24 option, 3.0r2 uses 2.4.20 with the bf24 option). A second, but not trivial problem is that the Broadcom Gigabit NIC is not supported in these versions.

The solution is not very easy, but is possible. You need another (supported) NIC (I've used a 3Com 3C590) and a normal SCSI disk to connect to the integradet disk controller. An IDE disk is no solution because the bios does not support a boot from an IDE disk - only from an IDE CD-ROM.

I have asked for help in the german Debian users mailing list, and received some very useful answers. The succesful trick came from Thomas Reiß, but Thomas Kratz helped me too.

Shortly it works as follows (a detailed description is listed later):

install a SCSI disk on the integrated SCSI controller
install a supported NIC in a PCI slot
install plain Debian 3.0 on the machine
install sources for kernel 2.4.22 on the machine
configure the kernel for support of the Broadcom adapter and the Smart Array (5i) controller
install the new kernel and reboot the machine, connecting the network cable to the integrated NIC
partition the RAID disk and transfer the disk contents to the RAID disk, configuring /etc/lilo and /etc/fstab
remove the disk from the integrated controller and start the machine

I have a working kernel deb package here, so don' t need to build the kernel for yourself.

If you need to install the agents, after a succesful installation please see here: http://www.sk-tech.net/support/HPrpm2deb.sh.html.

I have received a message saying that the method I used was not so succesful, maybe because Alex used an IDE disk instead of the suggested SCSI disk. You'll find more details (in german) on http://forschi.de/debian-server.php.

Matti Paksula has contacted me writing that he had found another method to set up Debian on this (and other "tricky" machines). For this you need a second working Debian machine. The details can be found on his page: http://www.cs.helsinki.fi/u/paksula/misc/debian_on_proliant.html.

And maybe you'll find this (non-technical) page informative: http://opensource.hp.com/opensource_projects.html. It say simply that HP itself is using Debian internally and it's sponsoring the Debian project, both directly (paying developers) and indirectly (paying structures).

Detailed instructions

(mostly from Thomas Reiß)

- set the machine OS in the bios to "Linux"

- build the array with the bios utilities

- install a SCSI disk (with at least 500 MB) to the integrated controller (Adaptec AHA 3960D/AIC 7899D)

- install a supported NIC in a free PCI slot and connect your network cable there

- install a minimum Debian 3.0r2 system on the added disk

- modify your /etc/apt/sources file to include Adrian Bunk's backports for Debian:
deb http://www.fs.tum.de/~bunk/debian woody/bunk-1 main contrib non-free

- apt-get update

- apt-get install kernel-source-2.4.22

- configure your new kernel (if you don't know how to make such a thing please look here) to include support for the Smart Array 5xx and for the BroadCom Gigabit ethernet adapter. Please don't forget the support for the Adaptec 78xx and for ext3 or ReiserFS if you've used this. The AGP support should be removed. I've added NET_FILTER to because the dhcpd needed it.

- build your kernel and install it

- reboot you system (with the network cable attached to the integrated NIC because it's now eth0)

- partition the RAID array with cfdisk (using device /dev/cciss/c0d0)

- format your disks with mkfs

- mount the partitions of the RAID disk at /mnt

- transfer the disk contents to the RAID disk (using cp -av)

- chroot /mnt

- modify your /etc/lilo.conf file to point to the new boot (/dev/cciss/c0d0) and root (/dev/cciss/c0d0p1) devices

- modify your /etc/fstab to point to the new partitions (/dev/cciss/c0d0px instead of /dev/sdx)

- execute lilo

- shutdown the machine, remove the added NIC, disconnect the added drive and boot your system. It should work now. If not, you can use the installation disk as fallback system.

Hope this helps!

And this is the original (german) mail from Thomas Reiß:

From: thomas at mypoint.franken.de (Thomas Reiß)
To: debian-user-german at lists.debian.org
Subject: Re: Installation von Debian 3 auf einem Proliant ML350 und SA641 Controller
Date sent: Fri, 19 Dec 2003 10:37:33 +0100

Hallo Wolfgang Riedmann,

> Hallo,
>
> sorry, wenn ich gleich mit der Tür ins Haus falle:
>
> ich sollte Debian 3 auf einem Compaq ProLiant ML350 G3 installieren.
> Der Boot-Controller ist ein SmartArray 641-Controller, wo auch alle
> Platten hängen.

Ich hatte die gleichen Probleme wie du, konnte das aber wie folgt
lösen.

- Im Server "Bios" das OS auf Linux eingestellt.
- Bootreihenfolge eingestellt
- RAID angelegt
- eine IDE - Platte (oder wenn vorhanden eine SCSI-Platte) einbauen
- Debian 3.0 R2 booten
- Eine Minimal Installation auf die IDE oder SCSI Platte durchführen
- Boot Diskette erstellen lassen
- Server von IDE oder SCSI Platte booten
- folgendes in die /etc/apt/sources eintragen

# Adrian Bunks Backport für Woody
deb http://www.fs.tum.de/~bunk/debian woody/bunk-1 main contrib
non-free

- apt-get update
- apt-get install kernel-source-2.4.22
- kernelkonfiguration mit fest einkompilierten cciss Treiber (Unter
Blockdevices der Punkt Compaq Smart Array 5xxxx Support) erstellen
- Booten und schaun ob das RAID erkannt werden konnte (Das klappt IMHO
für diesen Kontroller erst ab Kernel-Version 2.4.21 bzw. dessen cciss
Treiber Version vernünftig.
Es sollte beim booten in etwa folgendes auftauchen:

cciss: Device 0x46 has been found at bus 2 dev 1 func 0
cciss/c0d0: p1 p2 p3
VFS: Can't find ext3 filesystem on dev cciss0(104,1).
reiserfs: checking transaction log (device cciss0(104,1)) ...
for (cciss0(104,1))

- das Device /dev/cciss/c0d0 mit cfdisk paritionieren und formatieren
(das neue / Rootverzeichniss am RAID; swap Partition)

- die neuen Partitionen mounten (unter z.B: /mnt)

- das aktuelle Rootverzeichniss mit cp -av / /mnt rüberkopieren (ohne
/proc, hier reicht es nur das Verzeichniss anzulegen)

- chroot /mnt
- /etc/lilo und /etc/fstab entsprechend des neuen Device anpassen

/etc/lilo.conf

lba32
disk=/dev/cciss/c0d0
bios=0x80
boot=/dev/cciss/c0d0
root=/dev/cciss/c0d0p1
install=/boot/boot-menu.b
map=/boot/map
delay=20
vga=normal
default=Linux

image=/vmlinuz
label=Linux
read-only

usw..

/etc/fstab (z.B.)

/dev/cciss/c0d0p1 / reiserfs defaults
0 0
/dev/cciss/c0d0p2 none swap sw
0 0
proc /proc proc defaults 0
0
/dev/fd0 /floppy auto user,noauto 0
0
/dev/cdrom /cdrom iso9660 ro,user,noauto 0
0

# Filesysteme unter LVM
/dev/vg00/lv_home /home reiserfs defaults 0
0
/dev/vg00/lv_samba /var/samba reiserfs defaults 0
0
/dev/vg00/lv_opt /opt reiserfs defaults 0
0

- lilo aufrufen
- neu booten

So habe ich es zum laufen gebracht.
Leider habe ich die Tools für die RAID Configuration/Überwachung und die
Servertools noch nicht zum laufen bekommen. Ich hatte in der Arbeit mal
einen Link der Beschrieb wie man das machen könnte, allerdings bin ich
noch nicht dazu gekommen das auszutesten.

Es wurde ja auch schon beschrieben wie du eine Boot CD erstellen kannst.
Ich habe damals etwas entnervt aufgegeben, da ich die ganzen
Zusammenhänge etwas verwirrend fand.

Ich hoffe du kommst zumindest hiermit auf die schnelle zurecht.

Gruß
Thomas