I’ve been slowly moving along in this self-hosting journey and now have a number of services that I regularly use and depend on. Of course I’m backing things up, but I also still worry about screwing up my server and having to rollback/rebuild/fix whatever got messed up.

I’m just curious, for those of you with home labs, do you use a testing environment of some kind that you use, or do you just push whatever your working on straight to “production”?

  • InvertedParallax@lemm.ee
    link
    fedilink
    English
    arrow-up
    10
    ·
    4 months ago

    In my job? Yes.

    At home? God no.

    I make sure I can recover data when things go wrong, but otherwise my recovery path is redeploying quickly.

  • Avid Amoeba@lemmy.ca
    link
    fedilink
    English
    arrow-up
    7
    ·
    4 months ago

    Sir, every professional developer knows there’s never time and people to maintain the testing environment so testing is done in production! That testing environment you’re dreaming of is missed shareholder value.

    • pulsewidth@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      4 months ago

      Most importantly - the time and people = money.

      My last job had a dev, UAT, and prod environments because they knew it was important enough to the business to pay for them.

      I dont pay me anything for running my home environment - so, there is only production. And lots of backups.

  • JovialSodium@lemmy.sdf.org
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    4 months ago

    Nope. I fiddle until it does what I want. If the thing I’m working on is complex or I’m struggling with it I’ll keep versions of configs. And I back up working configs via an rsync job. Which isn’t a particularly robust solution but I’m content with it for my needs.

  • r0ertel@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    4 months ago

    After breaking “prod” many times, I have a Dev (local machine), Test (small VM) and Prod (big VM). My test is just less RAM and space and I need to spin down certain K8s things to spin up others, but it’s a close mirror of Prod, just less.

  • irotsoma@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    1
    ·
    3 months ago

    At work we have 6 environments other than production. At home just one. I created a way to ease deployment of the environment from scratch using a k0sctl config and argocd and the data gets backed up regularly if I need to restore that, too.

  • notabot@lemm.ee
    link
    fedilink
    English
    arrow-up
    1
    ·
    4 months ago

    I manage all my homelab infra stuff via ansible and run services via kubenetes. All the ansible playbooks are in git, so I can roll back if I screw something up, and I test it on a sacrificial VM first when I can. Running services in kubenetes means I can spin up new instances and test them before putting them live.

    Working like that makes it all a lot more relaxing as I can be confident in my changes, and back them out if I still get it wrong.

  • sugar_in_your_tea@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    4 months ago

    I test stuff on my laptop or desktop and then push it to my NAS. Everything is also containerized and snapshotted, so risk of breaking anything is pretty small.

    Also, I’ll have it run on my boot drive in “prod” for a bit before exposing it to my RAID, just in case there’s some weird issue on shared data. Switching it is just a copy + changing the container volumes.

  • N0x0n@lemmy.ml
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    4 months ago

    Production is my testing lab, but only in my homelab ! I guess I don’t care to perfectly secure my services (really dumb and easy passwords, no 2fa, not hiding plain sight passwords…) because I’m not directly exposing them to the web and accessing them externally via Wireguard ! That’s really bad practice though, but any time soon will probably clean up that mess, but right now I can’t, I have to cook some eggs…

    There are 2 things though I actually do have some more complex workflow:

    • Rather complex incremental automated backup script for my docker container volumes, databases, config files, compose files.

    • Self-hosted mini-CA to access all my services via a nice .lab domain and get rid of that pesky warning on my devices.

    I always do some tests if my backups are working on a VM on my personal desktop computer, because no backup means that all those years of tinkering for nothing… This will bring up some nasty depression…

    Edit: If have a rather small homelab, everything on an old laptop, still quite happy with the result and works as expected.

  • vegetaaaaaaa@lemmy.world
    link
    fedilink
    English
    arrow-up
    1
    ·
    3 months ago

    My prod and testing environments are 2 libvirt VMs on the same hypervisor. They run the same services, deployed and managed by ansible. The testing VM just gets less disk/CPU/RAM resources, and is powered off most of the time. Simple config changes? Straight to prod. New feature, risky change? Testing first.