Do ASP.NET Web Applications play nice with Fly.io?

Fly.io is a compelling option for hosting your web app.

The home page promises a simple process for launching and scaling, using commands like this:

fly scale count 3 --region ams,hkg,sjc

With that your app would be running on three instances in Amsterdam, Hong Kong, and San Jose (California).

Fly.io automatically routes traffic to the nearest deployed instance, and can take care of things like autoscaling (increasing the number of machines in use under heavy load).

But how does it hold up in practice and does it work with .NET web apps?

To find out I took a small, simple .NET 8 Blazor app and tried to get it up and running on Fly’s free plan.

Install the Fly CLI

The first step is to install the Fly CLI, as nearly everything you do with Fly uses it.

Here’s the command if you’re using Windows.

pwsh -Command "iwr https://fly.io/install.ps1 -useb | iex"

For other OS’s check out the docs here.

With that installed you need to authenticate with https://fly.io, which is done via this command:

fly auth signup

This takes you to the Fly homepage to set up an account.

It seems Fly are currently tweaking their plans/pricing and you do need to enter credit card details to create an account, but you’ll either be on a free plan, or be awarded free credit.

Either way, you should be able to try the rest of the following steps for free.

After following the steps you’re eventually returned to your command line.

It seems the easiest way to get an app deployed is to let the Fly CLI do it for you. The precise steps vary depending on how many projects your .NET app has.

Single Project Apps

In my case I’d spun up a new Blazor app using .NET 8.

The app isn’t using Blazor WASM for interactivity so has just the single .NET project.

So long as there’s only one project it seems the next step is pretty simple.

CD to the directory containing your Blazor app and type the following command:

fly launch

You’ll be presented with a number of questions.

I’ve given the app a name, and selected London as the region…

Fly offers the option to install a Postgresql database, and/or an Upstash Redis database.

I chose no for both.

Finally, when asked if I wanted to deploy now, I said yes.

Fly set about deploying the app and, a few moments later, it reported that it created 2 “app machines”.

It turns out, that was going to be a problem, but more on that in a moment…

Looking at the solution folder, Fly added a Dockerfile and fly.toml config file.

The config looks like this:

app = "net8test"
primary_region = "lhr"

[build]

[http_service]
internal_port = 8080
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
processes = ["app"]

The Dockerfile should look something like this:

# Adjust DOTNET_OS_VERSION as desired
ARG DOTNET_OS_VERSION="-alpine"
ARG DOTNET_SDK_VERSION=8.0

FROM mcr.microsoft.com/dotnet/sdk:${DOTNET_SDK_VERSION}${DOTNET_OS_VERSION} AS build
WORKDIR /src

# copy everything
COPY . ./
# restore as distinct layers
RUN dotnet restore
# build and publish a release
RUN dotnet publish -c Release -o /app

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:${DOTNET_SDK_VERSION}
ENV ASPNETCORE_URLS http://+:8080
ENV ASPNETCORE_ENVIRONMENT Production
EXPOSE 8080
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT [ "dotnet", "FlyDeployMultiple.dll" ]

Multiple Project Apps

I decided to check what happens if your app has multiple projects.

Turns out it you need to tweak the process slightly.

Let’s say you’ve spun up a Blazor project with interactive server and interactive web assembly enabled.

If so you’ll find yourself with a project structure like this:

Note the extra .Client project for any components which need to run in the browser via WASM.

To make Fly play nice with this structure we can start as if we’re dealing with one project.

From the main (server) app in the solution I ran the fly launch command (exactly as before).

CD FlyDeployMultiple
fly launch

• Gave the app a name
• Chose a region
• Selected No when asked to set up a Postgresql database
• Selected No when asked to create an Upstash Redis database
• Selected Yes when asked to deploy now

This time, however, I run into errors because client project could not be found.

#0 5.667 /src/Program.cs(1,25): error CS0234: The type or namespace name 'Client' does not exist in the namespace 'FlyDeployMultiple' (are you missing an assembly reference?) [/src/FlyDeployMultiple.csproj]
#0 5.667 /src/Components/_Imports.razor(9,25): error CS0234: The type or namespace name 'Client' does not exist in the namespace 'FlyDeployMultiple' (are you missing an assembly reference?) [/src/FlyDeployMultiple.csproj]  

This is because the build is using the current folder as the working directory, and it can’t find the related (client) project (which is a sibling in the directory structure).

The easiest fix here is to move fly.toml file up to the parent directory.

mv fly.toml ../

We also need to move the .dockerignore file to the same location.

Note, we don’t want to move the dockerfile, just the .dockerignore.

mv .dockerignore ../

Now fly.toml and .dockerignore are sitting at the same level as the solution, and project folders (as in the example below).

Almost there, now to help Fly out by directing it towards the Dockerfile in the main (server) project folder.

fly.toml

app = "net8multipleprojects"
primary_region = "lhr"

[build]
  dockerfile = "./FLyDeployMultiple/Dockerfile"

...

After making sure the terminal was still pointing at the root folder for the project (the one which now contains fly.toml) I attempted the deploy again.

fly deploy

This time the deploy went through just fine.

Growing pains

Great, now I could visit the app at its shiny new URL.

But we have a problem.

On reviewing the browser console, there seemed to be a lot of errors:

Doesn’t look great - so what’s going on?

There are two problems at play here and they’re both because Fly is running multiple machines for our app.

The first is to do with ASP.NET’s built-in AntiForgery protection.

The second is a Blazor Server problem.

Let’s take them in turn.

Errors validating antiforgery tokens

Faced with these errors I checked the logs in Fly’s control panel and saw errors like these.

[info] fail: Microsoft.AspNetCore.Antiforgery.DefaultAntiforgery[7]
[info] An exception was thrown while deserializing the token.
[info] Microsoft.AspNetCore.Antiforgery.AntiforgeryValidationException: The antiforgery token could not be decrypted.

So what is this DefaultAntiforgery and why is it failing?

If you look at a .NET 8 app’s Program.cs file you’ll find this line.

app.UseAntiforgery();

This is our clue as to what’s going on.

This line sets up the plumbing to ensure incoming form data is validated as originating from a form on your site (and not some third party trying to hack your app).

Blazor Server also relies on this Anti-Forgery mechanism when opening up connections with the client (for components running in interactive server mode).

The problem here stems from fly’s attempts to balance the load between multiple machines.

With multiple instances of the app running, incoming web requests can be routed to either one.

This is great from a scaling perspective, as you can easily spin up multiple machines in different regions to handle load from different parts of the world.

However this load balancing can cause issues for the anti-forgery plumbing in a .NET app.

To issue (and validate) anti-forgery tokens .NET relies on machine keys.

In a load-balanced environment users can find themselves with tokens issued from one machine, only to jump to another machine for subsequent requests, at which point the original token can’t be validated (because the machines have different keys).

Thankfully .NET has built-in support for storing keys centrally, in a location where all the machines can access them.

There are a number of ways to achieve this.

If your app has a database you can store them there, or you can store them on shared disk storage.

Here’s a list of storage providers in the official docs.

In my case I decided to use Redis to store the keys. If you’re interested in the specifics check out this link:

Spin up a Redis instance using fly, then configure it to store the keys.

However, even with that all set up and Anti-forgery keys stored centrally, there’s another, potentially bigger problem.

Blazor server load balancing challenges

I set the Weather page on the Blazor app to run interactively using Blazor Server.

Problem is, if you’re using Blazor Server load balancers are potentially challenging.

From the official docs on Blazor Server:

“SignalR requires that all HTTP requests for a specific connection be handled by the same server process. When SignalR is running on a server farm (multiple servers), “sticky sessions” must be used”

The Blazor team are pretty explicit that, for Blazor Server to work properly, you need to use sticky sessions (in a load balanced environment).

Sticky sessions enable a load balancer to identify requests coming from the same client, and always route them to the same server.

This is important for Blazor Server because the DOM’s current state for each client is stored in memory on the server. If a client gets disconnected from its original server, its state is lost.

Unfortunately, after a bit of research it seems Fly.io doesn’t have support for sticky sessions at the time of writing.

They may implement it at some point, but in the meantime the most practical solution appears to be to to limit your instances to one per region.

By sticking with one machine per region, it’s highly likely that a client’s requests will all go to the same machine (and unlikely that they would get routed to a machine in a different region).

If you’re not using Blazor Server or SignalR this probably isn’t an issue, but if you are, limiting your instances to one per region feels like a pragmatic, if slightly limiting workaround.

Closing thoughts

For a lot of .NET apps Fly seems like a compelling choice.

Its hosting costs are affordable, deployment is easy, and you can also host your Redis and/or PostgreSQL databases with them.

However, you do need to be aware of some potential issues if you decide to scale your app out to multiple machines.

If you have more than one machine per region you’ll run into issues using Blazor Server and/or SignalR.

The alternative is to scale up (have one machine, but make it bigger) or stick to one machine per region and deploy to more regions to get your machine count up.

Fly supports a lot of regions, so you can still scale your app out to many machines even if you’re sticking to one per region.

On a broader note .NET 8 makes it possible to use Blazor server sparingly, only enabling it for components which need to run interactively.

The side-effect is that your socket connections aren’t open for as long as they used to be in .NET 7 and earlier.

Now the connections are opened when needed, and closed again when users move away from a page which has interactive server components.

That might mean services like Fly.io have a better chance of working well for Blazor apps, as you generally don’t need the same socket connection to stick around for the entire time a user is interacting with the app.

If you’re using Razor Pages, MVC, or Blazor without the need for server interactivity, Fly.io is well worth a look.

If you’re dependent on Blazor Server, there may be simpler options elsewhere, which have native support for sticky sessions and thereby handle the challenges of load balancing in a more reliable way.

Next up