[post in progress]
Promise: this post will not be another rewrite of the official documentation. Here I will deal with issues you might run into but no one of these people actually did because they’re not actually using it.
The company I work for has a very simple need: display on large monitors some data on a web page with huge size fonts in full screen.
Currently we have 20 very old machines, bought as thin clients for VDI and then, when their flash disk (think of an SSD actually slower than a spinning HDD) got fried, converted to netboot via PXE. Not difficult as the CPU is an x86 VIA Eden. The PSU though are a bit custom and only last ~3 years. We bought 20 spare PSU in 2013 and another 20 in 2016; it’s now 2019 and the PSUs for the now obsolete thin clients are out of production. Time for a change.
The perfect solution is indeed a Raspberry Pi 3 B+ with PoE HAT – no PSU to fry, netbootable out of the box (with those pesky netboot bugs solved, such as the inability to boot from a TFTP server on a different network), no microSD to take care of, central management (being NFS rooted), small and nice.
First caveat: don’t be a cheapskate, buy the right case. The cost is >3x, but the absolute value is not that much – you can afford 17€ vs 5€.
Second issue: don’t buy a Pi 4 yet: the dramatic changes in the internal structure of the SoC make it so different that the bootstrap firmware is now on an internal 512Kb EEPROM rather than on the boot flash/TFTP server and had to be completely rewritten. However it is still (as of October 2019) lacking the netboot code. Of course you can boot just the kernel and cmdline.txt off a microSD ad NFS mount the root fs, but you’ll end up needing a microSD anyway so part of the advantages in netbooting it would be gone.
Let’s start.
Our x86 thin clients boot off initrd – actually an initramfs – which is nice since the client becomes completely independent from the network, but eats up precious runtime RAM and has size limits. So we decided to go for readonly NFS, with sensible writeable private parts overmounted on tmpfs. The backbone for the old x86 clients is based on DSL; on rPi we’re using dietPi.
Now, if you, like me, live in a structured network you might find that most “tutorials” for netbooting a rPi are nothing more than useless paraphrases of the sparse docs. The point is that you don’t and can’t have a do-it-all dnsmasq; you’ll probably have a centralized DHCP server with failover and relays, a TFTP server, an NFS server, and so on. So here’s some details about setting it all up.
Notes on setting up a NFS filesystem
Issue you’ll run into and how to solve them
DHCP on Windows
You can have a Windows DHCP server in lieu of the dnsmasq server or isc-dhcpd all tutorials suggest. This is rather frequent in the real, corporate world. Nothing really fancy to do here, you only need to set up option 66 as a string holding the IP address of the TFTP server. IP address, not hostname: the rPi bootload doesn’t have name resolving capabilities.
TFTP boot
Your TFTP server folder must hold the same content as the boot folder on your SD – that is the FAT32 partition that usually gets mounted under /boot – you can also mount it later but there’s no real need for that as the whole system will be readonly if you’re sane.
The actual files needed to boot a Pi are: bootcode.bin (except for the 4) (which can’t netboot yet anyway but whatever), start.elf, config.txt, (cmdline.txt
It seems there’s no real way to specify a boot path, ie to request files from a custom subfolder rather than from / – so to avoid me
NFS latency
If your NFS server is on a different subnet your NFS packets *will* be fragmented. Period. The default packet size is 8192 and there is no way a rPi can do jumbo frames. Handle fragmentation and your CPU usage will skyrocket. Plus your UDP packets will never go back and forth in the correct order, so retransmissions and timeouts are guaranteed. Just freaking lower the wsize and rsize values, on the kernel command line parameter nfsroot (after the path). You’ll have to modify cmdline.txt for that but I guess you already knew that. Maybe lowering the timeout could help as well, but I measured that just setting the NFS packet size to 1400 does wonders to your NFS performance.
PoE HAT fan
The fun part here is the whole devtree concept.
E a me? 