NFVPE Blog Posts

Ready to generate images using vLLM-Omni for inference, but with ComfyUI as the front-end? Here we go!

I’ve put together an improvement on a rad tool called canhazgpu. My buddy Russell built this tool to replace “ye ole spreadsheet” the team would use to reserve GPUs on a shared machine. It’s an awesome improvement for developers sharing GPUs – canhazgpu handles all the GPU allocation we used to track manually (on a, AHEM, spreadsheet if you didn’t catch that the first time). It’s primarily designed for a single host.

I’ve been pretty excited to say hello to vllm, which is a library for LLM serving and inference. That along with some recent upgrades in my lab setup, I figured it was time to do it in the way that best fit my own brand, with these parameters:

I had been putting off upgrading Fedora because… Wayland is becoming unvoidable. And I’ve been avoiding it for a couple releases now. And Fedora 42 is out. And since it’s the answer to life, the universe and everything, I can’t deny it. Wayland is the future.

A couple of summers ago my dad was telling me this story where a customer was in the show room of his sign shop. A customer’s kid, probably prompted by the CRT monitor and yellowed plastic of a late 1990s eMachines computer sitting on a desk in the showroom, announces…

So, are you familiar with DRA? Dynamic Resource Allocation (k8s docs linked)? It’s for “requesting and sharing resources between pods”, like, if you’ve got a hardware resource you want to use, say, a GPU in your pod, or maybe you’re, like, cooler than that and you wanna connect a smart toaster to your pods… Well, you could use DRA to help Kubernetes schedule your pod on a node that has a connected toaster, or, well, yeah a GPU. Popular for AI/ML or so I hear, have you? 🙂

In this article, we’re going to use LLaVa (running under ollama) to caption images for a Stable Diffusion training dataset, well fine tuning in my case, I’ve usually been baking LoRAs with the Kohya SS GUI.

We’re going to try to get some AI/ML workloads running using Podman Desktop’s new AI Lab Feature!

Today we’re going to get a demo for Kubernetes Networking Interface (KNI) rolling! If you wanna skip the preamble and get to the terminal, skip down to the requirements.

Time to automate some secondary networks automagically by having configurations produced using a large language model (LLM)!