Recent searches
Search options
Build system: a Makefile
CI/CD pipeline: the same Makefile, but running on someone else's server
Why are you making this so hard for yourselves? And no, you don't need Kubernetes. Yes, I'm talking to you. Just put it down already. Stop it. Stop it!
@tastapod ..."But I don't understand it, therefore it's magic."
Besides, the Agent that we use to build the Cluster could also run inside the cluster, so we save resources!
(And these are fairly tame examples...)
@tastapod does configuring the build process always take at least twice the time and effort of actually writing the code, regardless of platform and language, or is it just me
@toni @tastapod when I started with Rust, I was delighted that `cargo run` just works... ok, but just tweak build.rs to compile the static dependencies as optimised release but the code under development as debug... oh, embedded work for an ESP32? you want WSL then... oh and you'll need to figure out which dependencies from the Espressif framework you'll need to install...
it's still a whole lot better than rushing into C++ plugins for flight simulators without pausing to learn the fundamentals of how a dynamic library works, though
everyone always forgets the "learn" part of the line.
@mrcopilot @toni @tastapod I get knocked down, but I get up again, but I get knocked down, again, but I get up again
@jackeric It certainly does if you want it to!
@tastapod 100%, stop reenacting the maven build cycle in a build server file.
@tastapod I am 100% convinced that a better makefile is the key. Something that orchestrates all kinds of inputs and outputs, not just individual files.
A@samir I would love to see 'a better make'. I have seen plenty of contenders over the years, but nothing that nails the tangible-outputs-as-target-and-dependencies quite so well.
The challenge with orchestrating arbitrary (non-file) inputs and outputs is that you can't test so easily if something is 'done' or if it is older than any of its dependents.
You can orchestrate the whole thing with .PHONY targets, but then you have a different thing.
@tastapod I 100% agree.
There have been examples of attempts. Nix and Bazel, for example, use hashes rather than modified timestamps. (I'd argue that Bazel is the only reasonable successor to Make, and I'd argue even harder that no one should ever use Bazel, because it's impossible to use.)
I'm working on this in my small amounts of spare time. I've failed 5 times so far. Hoping 6th time's the charm!
(Yell if you would like to know more, but I will not inundate you without permission.)
@samir
> no one should ever use Bazel, because it's impossible to use
So much this! Simon Stewart tried a riff on this at Facebook called Buck, but it never seemed to catch on.
The tricky part is making ‘easy things easy and hard things possible’. As you say, the table stakes are high for even getting started.
@tastapod Absolutely, Buck was a nice idea (and Buck 2 is basically "Bazel but Facebook did it"; it even uses Skylark) but there's way too much friction.
I have a hypothesis that the solution to this is going to have to realise that a shell, a supervisor, a test runner, a build tool, a deployment tool, a spreadsheet… these are all special cases of the same thing. They may deserve different front-ends, but we might find that the underlying data model is better off unified.
It is nice to have the builds that run
(1) automatically,
(2) in a "clean" environment – one NOT full of all the random things that some developer has installed, and
(3) "in the background."
@JeffGrigg so write your Makefile to do that then!
- Go and Rust only know about what you tell them
- Python can use uv to build and run in super fast virtual envs
- At a pinch, your Makefile controls PATH, LD_LIBRARY_PATH, CLASSPATH, PYTHONPATH, etc.
If you are getting bitten by works-on-my-machine errors, your Makefile isn't done yet.
It boils down to this https://slsa.dev/spec/v1.0/levels
If I’m coding for fun then level zero is fine
If I’m doing it professionally and it needs to stand up to scrutiny/attack and the cost of failure is high, then level 0 looks pretty sketchy
And if I’m selling managed build tools level 3 looks both profitable & defensible when the smelly stuff hits the spinny thing
@JeffGrigg @tastapod You forgot (4) runs locally on developer machines. Pipeline servers are just automated developers with a standardised environment. They should build everything the *exact same way* as developers do locally.
@tastapod I am asking myself that same question every day now. No, every *hour*.
Pipelines have gone “enterprise”, and the inevitable enshittification has followed. And now orgs are baking that enshittification into their processes making it difficult to escape 
@thirstybear my malicious compliance is to write a Makefile that runs locally, then just have a single step in the 'enterprise' build sewage pipe that calls my Makefile.
Stick that up yer YAML!
@tastapod which make? Gnu or bsd :D. Just kidding but yes makefiles are great. You know use your npm maven cargo whatever but why cant you just wrap it in make so everyone can build it basically. Like make && make install done. Good old unix times :D.
@m3tti I try to make mine work with the antediluvian version on macOS, just for the lulz! But yeah, gnu make is my default.
@tastapod somewhat on a similar tangent: you don't need a fancy job scheduler, when a simple cron task firing a script or a binary can do the same thing.
@preslavrachev and in the long-running job world, I still haven’t found anything more simple or elegant than runit (sv, svdir and friends).
@tastapod A lot of Jenkinsfiles I work with actually could be shortened to (more or less) 'gradlew clean publish' / 'mvnw clean deploy' - they are more complex to distribute building, testing, publishing into different stages so that people can see where it fails more easily.
@tastapod For every existing build system I've ever touched, I've already written a boilerplate Makefile that solves the main problem that system was invented to solve... with the exception that my cross-platform Makefiles generally call cmake (I am aware of the irony of this reversal) to generate files for VS or similar, because why on earth would I ever want that pain? It's not like make is ideal, it's just that it's simple enough that I can't shoot myself in the foot with it alone, and I don't have to involve third parties in a single weekend project.
For the one time I've needed automated builds with results available from a single iteration for everyone? Just dumped output to a log on the webserver.