I’m in the process of wiring a home before moving in and getting excited about running 10g from my server to the computer. Then I see 25g gear isn’t that much more expensive so I might was well run at least one fiber line. But what kind of three node ceph monster will it take to make use of any of this bandwidth (plus run all my Proxmox VMs and LXCs in HA) and how much heat will I have to deal with. What’s your experience with high speed homelab NAS builds and the electric bill shock that comes later? Epyc 7002 series looks perfect but seems to idle high.
I’ve got a 3 node Proxmox/ceph cluster with 10G, plus a separate Nas. They are all rack mount with dual PSU. Add in the necessary switching, and my average load is about 800w. Throw my desktop (also on 10G) into the mix and it runs 1.1kw.
That’s roughly $50-60 extra in electricity costs for me monthly.
Would be around 300€ in Germany, on a cheap contract. Limiting myself to one combined NAS/application server atm, with the others turned on only if I want to try sth out.
deleted by creator
Average load 800W is 0.8kW24h30d=576kWh/M
Which is over 172€ on a 30ct/kWh contract.
Wow! I’m paying 10.5¢/kWh for electricity at home here in the US; it’s a little below the national average but not dramatically.
Yeah, we pay a lot. We also got one of the lowest downtimes regarding electricity, on average approximately 10minutes per year…so that’s kind of a (small) advantage you get for the premium price
I’m afraid of dumping 500+ watts into a (air conditioned) closet. How are you able to saturate the 10g? I had some idea that ceph speed is that of the slowest drive, so even SATA SSDs won’t fill the bucket. I imagine this is due to file redundancy not parity/striping spreading the data. I’d like to stick to lower power consumer gear but ceph looks CPU, RAM, and bandwidth (storage and network) hungry plus low latency.
I ran proxmox/ceph over 1GB on e-waste mini PCs and it was… unreliable. Now my NAS is my HA storage but I’m not thrilled to beat up QLC NAND for hobby VMs.
My 10G is far from saturated, but I do try and keep things using RAM where possible. I figure that with 100gb of DDR4 in my main server, that should be able to provide enough speed for a 10G link.
I’ve got ceph running on Intel Enterprise SSDs, so they are pretty quick.
I also tried running ceph on 1G. I found it unreliable as well.
I ise about the same. But that is more due to the hardware I got being a bit older. 2 dell R710s 1 R510 and a custom build server. Everything is still 1g. In my case electricity is not a big deal due to solar. We produce much more then we can use our self.
My pi costs probably around 20 a year lol.
Around 100 Watts for
- NAS with 4x3.5" HDD,
- Minisforum HM90 for Proxmox with 2x2.5" HDDs,
- 16 Port TP Link PoE Switch,
- TP Link router
- 2x Raspberry Pi 4b
But everything with gigabit speed. Doesnt need more at home
I’m running my smart home entirely from a single NUC running proxmox with VMs and LXCs for my services. It’s pulling ~7W on average
If you’re just running home automation, you do not need an Epyc 🤣
Get a low power anything to just run what you need.
I just moved my home assistant docker container to a new-to-me Xeon system. It also runs a couple basically idle tasks/containers, so I threw BOINC at it to put it to good use. All wrapped up with Debian 12 on proxmox…
(I needed USB support for zigbee in ha, and synology yanked driver support from dsm with the latest major version, so ‘let’s just use the new machine’…)
I looked at Epyc because I wanted to bandwidth to run u.2 drives at full speed and it wasn’t until Epyc or Threadripper that you could get much more than 40 lanes in a single socket. I’ve got to find another way to saturate 10g and give up on 25g. My home automation is run on a Home Assistant Yellow and works perfectly, for what it does.
Some unsolicited advice then: don’t go LOOKING for reasons to use the absolute max of what your hardware is capable of just because you can. You just end up spending more money 🤑
For real though, just get an N100 or something that does what you need. You don’t need to waste money and power on an Epyc if it just sits idle 99% of the time.
What I need is a 10g storage for my Adobe suite that I can access from my MacBook. I need redundant, fault tolerant storage for my precious data. I need my self hosted services to be high availability. What’s the minimum spec to reach that? I started on the u.2 path when I saw enterprise u.2 drives at similar cost per GB as SATA SSDs but faster and crazy endurance. And when my kid wants to run a Minecraft server with mods for him and his friends, I better have some spare CPU cycles and RAM to keep up.
Get a Drobo if you’re that worried about that kind of access then. Make it simple.
Otherwise anything with two NICs is the same thing.
5950x in an matx board with 15 x 3.5in drives 1 x sata sad 1 x optane u.2 drive (pulls like 10watts) 1 x Nvidia A2000 1 x Lsi 9305 16i 1 x 2.5gbe intel nic 3 x 140 mm fans at full tilt
Runs at like 120 watts at idle, like 220 watts with a good amount of work and peaks at like 320 watts if I make it do a lot of work
What in the world…
Dafuq you doing over there?
- Fujitsu motherboard
- Intel pentium G5600
- 6 HDD (4 x 4 TB 2 x 8 TB) spinned down
- 2 SSD for proxmox
- 6 CT and no VM for now
it runs at 16W mostly idle
My real server (Nextcloud/NAS/several more vm’s) uses 28 Watts on average. In addition, there is one Pi 4B running, and I don’t even know it’s wattage.
I’m planning on replacing the real server with a new one, with lots of cores and approx. 50 Watts then.
Pi4 tend to stick around 5w
I recently removed my 25Gbps PCIe dual port cards from my 2 servers because they were using 20W more. My entire rack including 2 UniFi PoE connections uses 90 W now (so 110 W just for having 25 Gpbs).
There is some heat from such cards, but usually it gets transported outside fine. The ones I bought did not come with a fan. I think you cannot operate them without one. The heat sinks get very hot.
I have a small setup for some self hosted apps and media.
- Beelink Mini S.
- 2 external 5TB drives.
- A USB fan used as an exhaust because the SSD inside gets a bit warm.
I think total power is about 30W.
Let’s see…
My servers (tiny/mini/micros) in total are about… 600W or so. Two NASs, about 15-20W a piece.
I spend a out $150/mo in electricity, but my hot water/HVAC/etc are the big power draw. I’d say about $40-50/mo is what I’m spending on powering the servers in my office.
Definitely puts off some heat, but that’s partially because it’s all in one rack, and I’ve got a bunch of other work hardware in there. It’s about 2 degrees warmer in my office than the rest of my home, but I also have air cycling all the time since it’s a single unit HVAC and I need to keep the air moving to keep it all the right temp in the other rooms anyway (AC will come on more often otherwise, even without my rack).
For my main server only… If HP iLO is to be believed, averaging around 130W.
Running: deluge, homarr, jellyfin, lidarr, navidrome, nextcloud, prowlarr, sonarr, whoogle and a minecraft server (VM) on TrueNAS Scale.
As for everything else (my router, switch and DNS/DHCP server, which is a separate machine, you can add another maybe 50W on top of it…
About 30 watts for a old Lenovo Thinkcentre with a i5-6500T and 8 GB RAM in combination with a DAS and 2x2TB HDD’s. I’m currently waiting for parts for my new server I’m building, a small N100 Mini-ITX board with 4x4TB HDD’s that hopefully has a similar power consumption.
I run a NUC11 so about 10W. 15-20€ per annum assuming a single tariff at 0.17€ per kwh. It can use up to 30W but only during heavy load which may be like 8 hours a week. But electricity is also cheaper during off peak hours so it averages to about that (we have 5 tariffs).
Load is NAS, media server, homeassistant and a usb zigbee router, *arr stack.
Power usage was my main concern and wanted something eco friendly.
From the wall I’m pulling 120w
Ryzen 5700G
128GB ram
2tb + 4tb NVMe drive
2 x 20tb HDDs
Unifi Enterprise 24 PoE
Mikrotik RB5009
2 access points
3 cameras
Fiber runs cooler than copper all of my SFP+ are fiber.
