What’s a “Thread Boundary” in Rust’s Async-Await ?

Async-Await paradigm is a great step forward in the evolution of programming. Not just it makes your asynchronous code “appear” synchronous, enhancing the readability, but also increases responsiveness and performance of the application, specially for the IO bound use-cases.

However, it can be a bit daunting to begin with, given the way it is dealt in different languages and frameworks!

Before I go any further, I will assume we have a basic understanding of Async-Await in Rust and know a bit on runtimes such as tokio, executors, futures and tasks.

Let’s start!

First, we will look at what it really means for a Future to be Send(Trait) under certain conditions.

Let’s take a very simple example.

Fig 1

The above example compiles fine and gives the desired output. We pass an async block to the executor. The async block within itself awaits on a sleep future to finish and runs to completion. All good!

Now let’s look at the following code. Here along with await we also have Rc.

Fig 2

This example fails to compile with the following error:

error: future cannot be sent between threads safely
   |
   |     let fut = tokio::spawn(async {
   |               ^^^^^^^^^^^^ future created by async block is not `Send`
   |
   = help: within `impl Future<Output = [async output]>`, the trait `Send` is not implemented for `Rc<i32>`
note: future is not `Send` as this value is used across an await

   |
   |     let r = rc::Rc::new(1);
   |         - has type `Rc<i32>` which is not `Send`
   |         tokio::time::sleep(time::Duration::from_millis(5000)).await;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ await occurs here, with `r` maybe used later

   |         println!("{:?}", r);
   |     });
   |     - `r` is later dropped here
note: required by a bound in `tokio::spawn`

   |
   |     T: Future + Send + 'static,
   |                 ^^^^ required by this bound in `tokio::spawn`

That’s a bit mouthful but elaborate. The compiler is basically telling us that async block needs to be Send because it contains Rc and as we know Rc is notSend . The error basically indicates that Rc may get accessed from multiple threads here.

But why does the async block needs to be Send just because Rc is Send? How on earth can Rc possibly be accessed from multiple threads here? It is not very clear by just looking at the code. The async block here does not hold any references to data outside it’s scope and appears as an “unbreakable” unit of execution for any thread to execute as a whole.

If we use a normal call from std library (non async-await) to spawn a thread and give a block of similar code, it compiles and runs fine. So what’s the difference?

Fig 3

The answer lies in the call to await and the async block can indeed be considered as a “breakable” unit of execution between multiple threads.

await itself is _the_ “thread boundary” here (See the message “future is not `Send` as this value is used across an await” in the error above) . Any code before the call to await and any code after it, might end up getting executed in two different threads by the executor. The await basically yields back to the runtime, so that the thread (which is free now) can be used to run something else that’s ready. So when this await is done (i.e future completed), the execution can continue in some other thread. More like work stealing.

Fig 4

Note: In the example above in Fig 3, where we spawn a thread using a std library call and give it a block of code, the thread gets suspended when we call sleep(). Of course, it too yields back — but to the OS, hence the thread cannot be utilized to perform any other work as it is not in a “runnable” state until the sleep() expires.

But how does a block of (async) code gets split among multiple threads?

Task vs Thread

The “thread” as we know it, is basically an OS thread — the one that std::thread::spawn() creates to execute a block of code. With runtimes like Tokio, it is called a “Task” which infact is a green thread managed by the runtime and not by the OS. It is a higher abstraction created by the runtime over the OS threads. Tasks drive the Futures to completion. They are built on top of the OS threads but not necessarily mapped 1:1 with the OS threads. This means an OS thread generally executes multiple tasks one after another, with very less overhead hence tasks are lightweight. On the other hand, context switching between OS managed threads (suspend & resume) is expensive. A task should avoid using blocking system calls as it can suspend the thread it is currently running on, preventing other tasks (and hence Futures) to run.

In Fig 4, the code blocks within the bigger async block are basically tasks that get scheduled on different threads.

Let’s run the following example:

Fig 5

Here we first create 5 futures (i.e 5 async blocks) where each future creates a sleep future and calls await on it. These futures run in parallel. Each block prints the thread id before and after await.

Here’s the output:

Thread 1 start, id = ThreadId(13)
Thread 3 start, id = ThreadId(9)
Thread 4 start, id = ThreadId(13)
Thread 5 start, id = ThreadId(9)
Thread 2 start, id = ThreadId(12)
Thread 2 end, id = ThreadId(2)
Thread 3 end, id = ThreadId(12)
Thread 4 end, id = ThreadId(2)
Thread 5 end, id = ThreadId(9)
Thread 1 end, id = ThreadId(12)

As you can see from the output, the tasks get scheduled on different threads.

You as a developer don’t have to worry about managing OS threads in your application. Runtime does it very efficiently and abstracts out all these complexities.

That’s it! Hope you enjoyed the article!