Loading…
Loading…
Why Aptibit built a native streaming engine for Visylix instead of wrapping FFmpeg. The architectural decisions behind handling thousands of streams per node.
When embarking on ambitious projects in the space of video streaming, particularly those demanding high-performance live streaming capabilities, a foundational question inevitably arises: "Why not simply use FFmpeg?" It is a fair inquiry, given FFmpeg's status as a cornerstone of the digital media space. This open-source multimedia framework is the backbone of countless applications, from global video platforms to desktop media players. Many commercial streaming solutions and video management systems are built upon its strong foundation.
However, at Aptibit Technologies, when we began developing Visylix, our enterprise AI video management platform, we opted for a different path. We chose to build our streaming engine from the ground up, rather than solely relying on FFmpeg's extensive but often narrowly focused capabilities. This decision was not made lightly; it represented a significant investment in time and engineering effort. The reasons behind this choice stem from the very nature of what we aimed to achieve: a platform capable of handling massive concurrency, sub-second latency, and integrated AI processing on every live stream.
Visylix is engineered for enterprise-grade AI video management, in estates where thousands of cameras connect to a single deployment, requiring live video with ultra-low latency ideally sub-second and the simultaneous execution of multiple AI models on each stream. This demanding environment operates 24/7, necessitating unwavering stability and minimal resource degradation.
During our evaluation of existing video management systems built around FFmpeg, we encountered a critical bottleneck. These designs hit a structural ceiling: each extra camera added another decode and re-encode cycle, CPU climbed toward saturation, live latency stretched to seconds, and stream drops became commonplace. This was not a limitation of hardware; it was a fundamental architectural constraint imposed by the way FFmpeg was integrated. The sheer volume of video processing required for so many concurrent feeds overwhelmed the traditional approach.
FFmpeg is, without question, a phenomenal tool. Its core strength lies in its ability to transcode media files, convert container formats, and manage individual streams with an unparalleled breadth of video codecs and audio encoders support. It excels at tasks like converting file formats, handling archive formats, and performing single-instance video recording. In essence, FFmpeg is a powerful toolkit, a collection of highly optimized libraries and a versatile command-line tool for media manipulation.
However, using FFmpeg for live video surveillance at the scale we envisioned means asking it to perform tasks far beyond its original design parameters. A typical FFmpeg-based live stream pipeline for such scenarios involves multiple decode-encode cycles: decoding the incoming stream, re-encoding it for storage, decoding it again for live viewing, and finally re-encoding it once more for browser-based delivery. Each of these operations is CPU-intensive. When multiplied by thousands of cameras, the cumulative CPU load becomes the limiting factor. Most video management platforms built on FFmpeg accept this limitation and advise customers to deploy more servers. Our objective was to create a solution that broke through this ceiling, not one that simply scaled out. Scalable architecture is what turns camera growth into margin rather than into server count.
The decision to engineer a custom streaming engine, particularly as a purpose-built native engine, represented a significant deviation from the easier path of wrapping FFmpeg. This approach added months to our development timeline and necessitated the reinvention of solutions that FFmpeg had already perfected decades ago, such as handling various video codecs and file formats.
However, this intensive undertaking granted us the unparalleled ability to design every layer of our system with a single, definitive purpose: to efficiently manage massive concurrent video streams, integrate sophisticated AI video processing, and minimize resource consumption. This deep-level control allowed us to architect a platform that could scale linearly. Three critical architectural decisions set our custom engine apart from traditional FFmpeg-based solutions.
A pervasive bottleneck in many traditional video streaming platforms, including those using FFmpeg, is the overhead associated with Input/Output operations. Each system call made for every I/O event adds up significantly. When managing thousands of streams, each generating dozens of I/O operations per second, this cumulative overhead can cripple performance.
To combat this, we developed a proprietary asynchronous I/O engine. This engine is engineered to process operations with near-zero overhead per individual operation. By drastically reducing the systemic cost of I/O, we removed the per-operation cost that caps wrapped pipelines, which is what lets one node carry thousands of streams. This forms the bedrock of our ability to handle more large videos and video recording tasks without performance degradation.
In a typical FFmpeg-based pipeline, video data is subjected to multiple memory copies as it navigates through various stages, such as decoding, AI analysis, and storage. At the stream counts a single node is expected to carry at 1080p and 30 frames per second, this consumes hundreds of gigabytes per second of memory bandwidth solely to move data around. This is inefficient and fundamentally limits scalability.
Our custom architecture fundamentally eliminates these unnecessary data copies. Video data flows smoothly through the entire pipeline without redundant duplication. This removes the redundant frame copies that dominate memory bandwidth in a wrapped pipeline and, crucially, liberates substantial CPU cycles. These freed-up cycles can then be dedicated to actual video processing and AI inference, rather than being consumed by mere data wrangling.
Standard memory allocators are designed for general-purpose applications, offering a balance of features for diverse workloads. However, video processing at scale is anything but general-purpose. Frames arrive at variable rates, AI models require dynamic memory allocations, and recording buffers must expand and contract fluidly based on event triggers.
We engineered a specialized memory management system precisely optimized for these dynamic patterns. In continuous operation, Visylix holds steady because its allocator is built for the frame patterns video produces, where general purpose allocators fragment over time. This specialized memory management is vital for maintaining predictable performance, whether handling live streams or complex video workflows.
The outcome of those architectural decisions is a large gap in efficiency. A single Visylix node handles thousands of concurrent streams at a fraction of the CPU that a traditional FFmpeg-based video management system needs for a few hundred streams on the same hardware. We deliberately do not publish a single headline capacity number: it depends on resolution, codec, frame rate and how many analytics run per stream, so we size and validate it on the customer's own hardware during the proof of concept.
Live view is sub-second over WebRTC, up to sub-200ms in ideal conditions. That contrasts with the multi-second HLS-based cloud viewing typical of VSaaS platforms. Our architecture also supports multi-cluster federation designed toward very large, city-scale estates, rather than concentrating everything on one node. This matters more every year as live video volumes keep growing.
Undeniably, constructing a custom streaming engine from the ground up demanded significantly more time and resources than simply wrapping FFmpeg. We had to meticulously address challenges in video codecs, container format handling, streaming protocols negotiation, buffer management, and a many of other complex issues that FFmpeg solves out of the box.
However, the reward was the creation of a platform that delivers capabilities unattainable by any FFmpeg wrapper: a system that carries thousands of streams per node in our internal testing and keeps live view sub-second. For our clients, this translates directly into fewer servers, reduced operational costs, and truly real-time video streaming that enhances their operations and user experiences.
Every startup in the video management and streaming protocols space faces the fundamental question: build or wrap? Many opt to wrap FFmpeg for a faster time to market. Our strategic imperative, however, was not speed to market, but long-term performance and scalability. We optimized for the critical moment when a customer connects their 1,000th camera, ensuring that performance does not just hold, but excels.
There is no substitute for building technology with architecture tailored to perform at scale. FFmpeg remains an incredible tool, indispensable for its intended purposes in video processing and format conversion, supporting a large array of video codecs and audio encoders. However, when a product requires handling thousands of concurrent live streams, integrating complex AI video processing, and achieving sub-second latency, an architecture explicitly designed for these challenges is paramount.
We built Visylix from the ground up in Kolkata. Today, it carries thousands of concurrent streams per node with sub-second live view, validated on each customer's own hardware during the proof of concept. That would not have been possible by merely wrapping FFmpeg. Sometimes the hardest engineering decision is the right one.
FFmpeg is brilliant as a transcoding library. It's not a streaming server. Wrapping it in a server process works fine until you hit a few hundred concurrent streams, then memory fragmentation, process-per-stream overhead, and lack of async I/O start destroying performance. For thousands of streams per node we needed ownership of the entire pipeline.
On identical hardware we handle substantially more concurrent streams at far lower CPU utilization than an FFmpeg-wrapped server pushed to its limits, with a much smaller memory footprint thanks to a custom memory allocator and shared buffer pools. Live view stays sub-second over WebRTC under load, up to sub-200ms in ideal conditions. We do not publish a headline streams-per-node number, because it depends on resolution, codec, frame rate and analytics load; we measure it on the customer's own hardware during the proof of concept.
Garbage-collected runtimes introduce latency spikes we could not afford at this scale. We chose a native systems approach with modern asynchronous I/O and a custom memory allocator, which gave us deterministic memory behavior and predictable performance under sustained load where it matters most.
Many months of focused work by a small senior team, and it continues with every release. That's an investment most companies can't justify, which is why so many VMS and streaming products ship on FFmpeg wrappers. For us, it was the difference between competing on features and competing on the underlying architecture.
Small deployments, batch transcoding jobs, recording pipelines, and any workload where latency and concurrency aren't the binding constraints. For those use cases FFmpeg is unbeatable. The moment you need thousands of concurrent live streams with AI analytics in the hot path, you're probably building custom.
No. It's proprietary to Visylix, though we draw on open standards (WebRTC, RTSP, HLS, SRT) and open-source libraries where they're best in class. The engine itself is our differentiator, not a commodity.