Project Greenhouse
Grid Hardware
Published: 2019-03-06

This is the development diary for Greenhouse, my attempt to build a DIY compute farm in a box. In other words, a multi-machine chassis.

2018-11-01 — Introduction and Plan

The core idea is to assemble individual compute nodes on trays, in the most compact way feasible, and then to stack nodes atop each other in an enclosure.

The components of each node will be:

  • A tray made from some thin, rigid, wood-based board.
  • Nylon washers/spacers and steel bolts, for standoffs and attachment of mobo to tray
  • A Mini-ITX motherboard with wifi and an M.2 slot
  • A CPU
  • 2x4G RAM
  • An M.2 SSD (cheap now, and eliminates 2 cables)
  • An SFX PSU
  • 2x 120mm fans in a →[fan→mobo→fan]→ configuration

This will be prototyped out using cardboard in place of a hard tray, to determine optimal placement and connection of components. Already established is that:

  • The mobo and PSU should be oriented toward a “side” of the case for external connections, since the entirety of the “front” and “back” will be intake/exhaust.
  • The PSU will be anchored to the tray with velcro
  • The fans will be anchored to the tray with zipties

After placement is worked out, a verfication test model will be built from hard board.

If that works, construction of the chassis will begin. It will be constructed of pre-drilled L channel. Once there is a chassis, conversion of existing compute nodes to tray builds will begin.

The biggest unknown right now is how to construct the chassis enclosure. But that can be solved along the way.

Don’t worry if this doesn’t all make sense. There will be copious photos once work gets underway.

2018-11-11 — Prototype Layout Mk I (Design)

Won’t be fully testing this out tonight because I got #8 screws instead of #6, which are ever so slightly too big for the holes in the motherboard. This is the first layout I’m going to try though. Also I forgot to pick up an M.2 screw (2mmx3mm). I should probably switch to metric screw sizes all around.

Layout Mk I

The base dimensions are 44x23cm. That’s 1.012L for every cm of height. I’m using 120mm fans, so the absolute minimum height of one rack unit will be 13cm (so: 13.156L per U). There’s no way I can pack them in that tight in the real world, however. I’ll need 2-3cm for under-tray cable management, and probably another cm of height for the tray itself.

Actually, I take that back. After a few minutes of thought and research, I might be able to use a tray board which is 3mm thick. Or if that’s too flexible, there is also a 4mm option.

As for cable management, up to this point I had been imagining some sort of mesh strech netting. I now believe that’s dramatically over-doing it, and the correct answer is to simply cut some slots on either side of the tray and put loops of velcro through them, to cinch down the small number of cables that need to run under each tray.

2018-11-25 — Prototype Layout Mk I (Test)

After 2 weeks of failing to acquire an M.2 screw, I found some today. Then I (spoiler: attempted to) hook everything up and immediately discovered that the Mk I layout is a failure.

I initially guestimated cable lengths by running over the mobo, and in this configuration everything reached with plenty of length to spare.

But when I ran cables under the cardboard tray, the CPU power cable was just barely too short, and the ATX power cable was drastically short. I forgot to account for the length used in bends and the length used up by the spacers under the mobo.

Tomorrow I’ll revise the layout and try again.

The good news is that #6 screws and #6x3/4 spacers work great. I’m sticking with SAE screw sizes because the selection of small-diameter metric screws available to me is very limited – and nylon spacers even moreso.

2018-12-31 — Layout Mk II Thoughts

I haven’t done much work on this, but I’ve been doing a lot of thinking on it. Here’s how things are looking right now.

The trays for the Mk II design will be much smaller. This is because the PSU is going to be slung under the tray, and beacuse the 120mm fans won’t be on the tray at all.

The big problem with the Mk I layout was that the PSU cables used up a lot of run-length dropping down under the tray, then coming back up to meet the PSU. I’m using SFX PSUs, which are tiny and high quality, but come with shorter cables. It took a long time to come to the realization that the PSU could be velcro’d under the tray just as well as it could be velcro’d to the top. This actually makes several things easier.

  • It solves the cabling problem
  • It takes care of an alignment issue, where the PSU’s rear needs to be flush with some edge of the chassis, for cabling and venting purposes
  • It solves the problem of the PSU’s positioning possibly creating large amounts of empty space on the trays, because now the PSU will be slung under the mobo.

The fans are being moved from the trays to the sides of the chassis. They might be aligned with the middle of each tray, to provide over/under airflow, but they might be aligned with the trays, so that the pieces of L channel holding the trays don’t block airflow.

This should get tested in the next 2 weeks.

2019-01-20 — Layout Mk II

Today I bit the bullet and put my abominable carpentry skills to the test.

Here you can see the Mk I prototype tray next to the Mk II template tray.

Prototypes Mk I and II

Close-up of the template tray. Did I mention that I live in a 1BR apartment, and have no woodworking skills or tools? Or a workshop?

Anyway, the single outer holes are the mount points for the motherboard. The drilled-out slots are where the suspension system for the PSU attaches.

This tray trades area for height. It’s 9”x7”.

The PSU suspension is very high-tech, being made of two kinds of Velcro. Super heavy-duty adhesive strips go on the PSU itself.

The PSU will then be strapped to the tray by two long strips of double-sided velcro. You can see the alignment in this fitting shot.

And here’s everything anchored in place.

Now the tray gets flipped, and the mobo goes on top. It’s mounted the same way as in the Mk I tray, with the same hardware. The spacers don’t need to be 34” though — 12” will be fine, and will be used once I move to the next phase.

Top view of it all put together. There will be some tweaks to the tray for cable management, but that’s a minor thing to work out now.

The stack happens to stand 7” tall. That works out to 6.8L per unit (almost exactly half of the Mk I design). I don’t know what the spacing will need to be yet, but I’m hoping to come in under 30L for a 4-tray chassis.

2019-02-13 — Chassis construction

With the tray design worked out, the past three weeks have been devoted to working on the chassis. I have learned a lot. I have learned exactly how bad I am at metalworking. I have learned that I could build another one of these in 1/10th the time (or less). But mostly I have learned that I do not want to do this again, at least not for a good long while.

Things started very simply. This is the very first cut and fit test. It’s one of the bottom crossbars which act as feet and hold the sets of uprights apart. Attached to it are two of the four vertical uprights of L-channel.

After that I cut the other end pieces, and a set of eight smaller crossbars for holding the trays. That was all pretty straightforward. Then the trouble began: I needed to cut out notches so that the trays could slide in and out of the chassis. Below is the first attempt. With the simple hand tools I had available, this approach wasn’t working very well at all. This incomplete cut is around 20 minutes of work.

This was so frustrating that I left it alone for some days until I came up with a solution: a sequence of simple cuts, and use of pliers to flex the pieces loose at the L-bend (yay metal fatigue?)

This worked well. It was more cuts, but each was small, and perpendicular to the piece, so the work went much more quickly. Before long, I was ready to test the trays for fit.

Everything was great, except that the tray didn’t fit. The tray itself was slightly too wide, and there wasn’t enough clearance in the notches for the motherboard to clear. In my mind the mobo was set in from the edge of the tray far enough that clearance with the L-channel wouldn’t be a problem. This was almost the case, but not quite.

This was also very frustrating, and, combined with the upcoming arrival of a visitor, I ended up setting things aside for another week. Afterward, I made another round of cuts to produce large, clear, rectangular cutouts for the trays.

But before this, I had disassembled the other three compute nodes, thinking that I would be slotting them into the chassis in a few hours. That didn’t work out, so the top of my dresser looked like this for a week:

2019-02-17 — Assembly

Finally got back to it today. Cut the remaining 3 trays (adjusting the width down slightly), and put everything together. It all went smoothly.

Here’s the stack. It’s a meter high, the side you’re looking at is about 22.5cm, and the shorter side is about 18cm. The whole thing is 40.5L.

Three-quarter view. The trays with light-up HSFs are Ryzen 7 2700s. The other two are R5 1600s. Other than that the machines are identical.

A look at a tray, with the over/under mobo and PSU arrangement and push-pull 120mm fans. Temps are stable, swinging a few degrees in the 58C to 60C range, while the CPUs have been at 100% utilization for hours.

You can see that I could easily have shortened the inter-tray spacing by one step, and everything would have still had clearance and airflow. It’s suboptimal, but I’ll take it.

And that’s it, until this summer when the Ryzen 3000s come out!

Bill of Materials

  • System
    • Mobo: Asrock B450 mITX
    • CPU: Ryzen 27001600
    • PSU: Corsair 450W SFX
    • RAM: 16Gx2 DDR4 2666
    • SSD: M.2 128G
    • Fans: 2 X Arctic F12 120mm
  • Tray dimensions: 8 1/2x7”
  • Tray material: 14” plywood
  • Mobo attachment (per tray):
    • 4 X #6x1 14” screws
    • 4 X #6 nuts
    • 4 X #6x1/2” nylon spacers
    • 8 X #6 nylon washers
  • PSU attachment: doublesided velcro tape; adhesive velcro tabs
  • Chassis material:
    • 4 X 1.25” slotted steel angle, 3’ (uprights)
    • Addl. 1.25” slotted steel angle (crossbars: sufficient for 3 X 9", 8 X 7")
  • Chassis fasteners:
    • 22 X 1/4x1/2” hex bolts
    • 22 X 14” nuts
    • 44 X 14” washers