Content Delivery on Roblox

Imagine that you are a playwright—or any type of person who creates some abstract product for an audience. The relationship between creator and consumer is dependent on the ability of there being some means to consistently and steadily deliver the product created to the consumer waiting. Minimizing the wait time and ensuring smoothness while delivery may be in order is a top priority.

For Roblox creators, the idea that we are responsible for figuring out how to deliver our content to our consumers is foreign—Roblox's engine handles that automatically. In this essay, I will detail why creators should familiarize themselves with this responsibility, and how they can use techniques to optimize their workflow in relation to it, taking control and optimizing the content of their experience for the Roblox engine.

At a broad scope, the built-in Content Delivery Network ("CDN") of Roblox is effective. It powers millions of experiences, delivering millions of gigabytes in content straight to consumer machines every day. All while requiring 0 input and configuration from the massive pool of creators. With this generalization however, comes a drawback. There is no surefire way to configure the Roblox CDN and optimize its delivery for your experience's users. This becomes a problem at scale, when an experience is comparable in asset scale and approaches aggregate content sizes greater than, let's say, 5-10GB worth of pure cloud content like animations, images, sounds, etc...

Most experiences do not reach that level of content. Some might argue that this engine is not prepared for that size of content. I have things to say regarding the engine's scalability and feasibility regarding aggregate content at scale. But that is not the topic of this essay. What I will be covering is how creators can both plan and adapt existing experiences for this kind of content scale. The specific approach is optimizing the delivery and client ownership of a divided number of assets.

So, how would one create such an optimization? Wouldn't this require engine-side access which unprivileged users like developers do not have? The answer is no. We can optimize this, but it requires some tricks and it also requires some prep. It also relies on developers trusting one another and agreeing to some standards inside your operational boundaries (your studio.)

The first condition is that you store assets on the server exclusively. This is the backbone of your CDN. Some operational actor will then exist on the server. This actor, which we will call the CdnDeliveryManager, will listen for requests. What kind of requests, you might ask?

The second condition is that you store references on the client to assets in some standard way. Then, some operational actor will exist on each client. Those client actors will be the interface, like the concession booth at a movie theater handing out goods in exchange for money and a request. This will be referred to as the CdnClientCaller.

If players are the moviegoers, and assets are the concessions, then the CdnDeliveryManager is the person who drove a truck full of candy to the theater, and the CdnClientCaller is the underpaid minimum wage worker who gives you your overpriced concessions. I think this analogy is quite good, if I do say so myself.

Why would you take this approach in the first place, though? Couldn't you just store the asset directly on the client in say, ReplicatedStorage, so that when they boot up the experience it is loaded automatically?

The reason we don't do that is the same reason movie theaters don't turn their concession stand into a food shelf. If all of those assets were shoved into the client and streamed in automatically, there is no granular control over the dynamic downloading of assets and asset ids. Developers are only ceded 1 control method over the dynamic downloads, that being ContentProvider:PreloadAsync() What's worse, this method only changes the priority of the Roblox CDN. Meaning, it doesn't guarantee that some asset will be downloaded, and it doesn't stop other assets from being downloaded.

What we need is a concession stand. We need a way to ensure that only the necessary assets are downloaded and loaded, and only at the correct time. Enter our CDN solution. With this approach, some asset may be exclusively loaded to the client, while others may be exclusively stored and ignored as they are only stateful on the server. In reference form, we gain a lot of efficiency for not having excessive assets being streamed in the background. We can also provide more finite solutions to asset loading, such as inclusive or exclusive asset loading patterns and legitimate downloading methods exposed to unprivileged users like developers.

Roblox's dynamic asset streaming solutions are unmatched in their innovative and powerful utilities. However, it is my opinion that they are underserving creators and failing to meet the technical demands and interests of creators and studios who are interested in scaling experiences to levels that are acceptable, technically feasible, and even standard outside of the Roblox ecosystem.

Making use of the solution described above in concrete form has lead me to create a number of Services (A Service is in reference to a singleton unit of logical scope, which maintains a domain of excellence and provides utility or work output for external logical game units. This is fancy language for "A Service is a thing that does stuff, usually in 1 specific area").

The first service which I will describe implements the use of a technique which I will call cdnKeys. clientCdnService is a stateful game singleton that encapsulates logic based on what was earlier referenced as the concession booth manager. It is responsible for accepting a ticket (A cdnkey) and return a concession (An instance asset).

The server counterpart to clientCdnService is merely known as cdnService. This service encapsulates the request endpoint on a remote machine, listening for valid requests outputted by a clientCdnService in the form of a clientCdnRequest. When a clientCdnRequest is received, the cdnService will perform a search through a pool of assets denoted as the assetPool. If a legitimate and securely accessible asset is found, then an instance of the asset will be passed over the remote boundary and returned to the clientCdnService. This is the basic logic pipeline of the CDN implementation using a cdnKey system in Roblox.

Using a system like this isolates assets, making them only accessible upon specific and legitimate request. There is a stipulation: When an asset is passed over the remote boundary and ceded to the clientCdnService, it will now be loaded on that client's machine for as long as the game state is running. There is currently no API to dynamically load or unload any type of asset on demand using the client state. My solution only adds a logical layer to Roblox's CDN, formalizing a RFC request process and prevent excessive or unwanted asset streaming as a background decision made by the Roblox dynamic CDN.

Roblox Engine is not designed for users who want control over the state of their users machines. In practicing this addition of a logic layer to the dynamic CDN, we are merely hacking our way around a more serious issue that is actually addressed in production-grade industry standard engines. It would also be technologically and/or financially unfeasible to expect Roblox themselves to re-organize their entire content streaming system, (for example, I recently thought to myself, "why doesn't Roblox allow a games-as-a-package system where they can just download all the assets at once and play the game without weird interruptions or jarring loading issues?")—Roblox is built around a certain infrastructure. Reorganizing that infrastructure is expensive, time-consuming, and requires technical expertise. The cost-benefit analysis which they have likely performed as far as restructuring their CDN has most likely informed them that it would cost significantly more than it would return for them to restructure the way creator's assets are delivered. That topic is perhaps the subject of another blog post, though.

In conclusion, I have presented a system which would add theoretical control and logic layers to a system which is normally beyond the scope of a creator's decisions. As developers, we focus on creating assets and creating the game. It is normally the duty of Roblox Corporation themselves to focus on such things, but at scale, these issues are commonly unaddressed by the Corporation and thus need community solutions.

—Marceline Croyle