Very neat visualizations! Really good way to demonstrate the underlying principles. Great article.
I will add one thing: the author makes much of how problematic CPU-intensive tasks are. That's true, but also: if you need to do a CPU-intensive operation, you need to tie up a CPU to do it, there's no getting around it. What you might want to do is break the computation up in to smaller chunks with `.await`s sprinkled in there or dispatch it to a background thread, but at some point you have to pay the cost in CPU time.
What is much more problematic is blocking I/O operations. That also ties up a CPU and blocks the event loop, but for no good reason. All the CPU is going to be doing is sleeping waiting for the network or hard-drives or whatever it is to do it's work, when it could be doing other useful things, processing other events. That kind of thing leads to huge under-utilization of your resources, and it kills your concurrency.
This means you really can't mix async I/O with blocking I/O: if you go async, ALL your I/O operations need to be async. Or you have to do it off the event loop in a background thread or something, but then you're losing all the benefits of using async in the first place.
I went through a phase of writing a lot of little things in async Python which was a lot of fun but I wrote an app that really tied up the event loop and now, at least for web servers, I develop with Flask, deploy to gunicorn and often front with IIS or nginx to serve up images and big files now.
One thing I still like async Python for is programs that have a number of polling tasks going on at a slow cadence, say once a minute or less, where it doesn't really matter to me if a task gets delayed 5 seconds because of another task. In that case writing a number of tasks that await.sleep is easy and fun and works great -- I'll use async io in the tasks if it is easy to do so but if I use blocking I/O and it blocks... no problem.
At work I've been writing web servers in Java (or C#) for a couple of jobs and between threads and the garbage collector I also think it is easy and fun. Sure if I was handling Google-levels of traffic I might think differently, but threads scale pretty well for 99% of the web sites out there. Java gives you all the right primitives to do common tasks: sometimes I code something up with an Executor in 20 minutes and it works right the first time and gets a 14x speedup with 16 cores, somebody else tries Actors in the language of the day and spends a few days on it, never gets more than a 3x speedup and gets a different answer every time. (Threaded programming is easy when you have a rich primitive library you get in trouble when you get it in your head that it is noble to build everything in scratch from a small set of primitives)
Java even has virtual threads that play well with real threads if real threads aren't good enough for you.
but in an async I/O server scenario, the typical architecture is a fixed-size pool of worker threads, which ideally entirely occupy a single core each. therefore, if one of those threads is blocked, that core is wasted.
> Unlike Node.js, Rust's Tokio allows us to spawn a new Task and run futures within it.
Nice article! For future reference in Node.js this would be `worker_threads` or if you want a higher level tokio like API something like https://piscinajs.dev/
This has been possible in Node.js for quite a while
--
Also for blocking I/O CPU bound parallelism I've found Rayon to be a pleasure. It has a few caveats but it leverages the Rust race condition checking well and it's very ergonomic.
I happily mix it with `tokio` in an application that is CPU bound but has to perform some async I/O (due to the SDKs it uses)
It’s never addressed in the article that the calculations running on spawned threads produce deformed sine curves. Is that just a result of how its plotted or do tokio threads have less accurate track of time?
Thats a good observation. I believe the deformation comes from the variable runtime of the sin method. Even with the 100 micros sleep that I've put there, in reality it takes a different amount of time between each data point calculation, making the sine wave a bit wonky. This gets worse as more and more calculations are done in parallel
> In other words, each tick of the underlying clock might not be the same length (e.g. some seconds may be longer than others). An instant may jump forwards or experience time dilation (slow down or speed up), but it will never go backwards
> Alright! We managed to plot two sine waves in the most convoluted method ever!
Those sine waves start to resemble a multi-voice music score. You might get some interesting pitch variability depending on other processes that are running.
This is obviously a troll comment, but it made me curious.
I'm familiar with Plan 9, and I know about the interesting mount namespace shenanigans you can pull off with it, but does it actually have anything that would help with async?
Very neat visualizations! Really good way to demonstrate the underlying principles. Great article.
I will add one thing: the author makes much of how problematic CPU-intensive tasks are. That's true, but also: if you need to do a CPU-intensive operation, you need to tie up a CPU to do it, there's no getting around it. What you might want to do is break the computation up in to smaller chunks with `.await`s sprinkled in there or dispatch it to a background thread, but at some point you have to pay the cost in CPU time.
What is much more problematic is blocking I/O operations. That also ties up a CPU and blocks the event loop, but for no good reason. All the CPU is going to be doing is sleeping waiting for the network or hard-drives or whatever it is to do it's work, when it could be doing other useful things, processing other events. That kind of thing leads to huge under-utilization of your resources, and it kills your concurrency.
This means you really can't mix async I/O with blocking I/O: if you go async, ALL your I/O operations need to be async. Or you have to do it off the event loop in a background thread or something, but then you're losing all the benefits of using async in the first place.
I went through a phase of writing a lot of little things in async Python which was a lot of fun but I wrote an app that really tied up the event loop and now, at least for web servers, I develop with Flask, deploy to gunicorn and often front with IIS or nginx to serve up images and big files now.
One thing I still like async Python for is programs that have a number of polling tasks going on at a slow cadence, say once a minute or less, where it doesn't really matter to me if a task gets delayed 5 seconds because of another task. In that case writing a number of tasks that await.sleep is easy and fun and works great -- I'll use async io in the tasks if it is easy to do so but if I use blocking I/O and it blocks... no problem.
At work I've been writing web servers in Java (or C#) for a couple of jobs and between threads and the garbage collector I also think it is easy and fun. Sure if I was handling Google-levels of traffic I might think differently, but threads scale pretty well for 99% of the web sites out there. Java gives you all the right primitives to do common tasks: sometimes I code something up with an Executor in 20 minutes and it works right the first time and gets a 14x speedup with 16 cores, somebody else tries Actors in the language of the day and spends a few days on it, never gets more than a 3x speedup and gets a different answer every time. (Threaded programming is easy when you have a rich primitive library you get in trouble when you get it in your head that it is noble to build everything in scratch from a small set of primitives)
Java even has virtual threads that play well with real threads if real threads aren't good enough for you.
> blocking I/O operations. That also ties up a CPU and blocks the event loop
They do block the event loop (if single-threaded) but they do not "tie up" a CPU (core), the OS will schedule other threads on it.
assuming you have other threads
but in an async I/O server scenario, the typical architecture is a fixed-size pool of worker threads, which ideally entirely occupy a single core each. therefore, if one of those threads is blocked, that core is wasted.
> Unlike Node.js, Rust's Tokio allows us to spawn a new Task and run futures within it.
Nice article! For future reference in Node.js this would be `worker_threads` or if you want a higher level tokio like API something like https://piscinajs.dev/
This has been possible in Node.js for quite a while
--
Also for blocking I/O CPU bound parallelism I've found Rayon to be a pleasure. It has a few caveats but it leverages the Rust race condition checking well and it's very ergonomic.
I happily mix it with `tokio` in an application that is CPU bound but has to perform some async I/O (due to the SDKs it uses)
> piscinajs
Lmao this wordplay made my day
> Rust race condition checking
Nit: Rust does not prevent race conditions, it prevents data races.
A tokio task is not a thread
It’s never addressed in the article that the calculations running on spawned threads produce deformed sine curves. Is that just a result of how its plotted or do tokio threads have less accurate track of time?
Thats a good observation. I believe the deformation comes from the variable runtime of the sin method. Even with the 100 micros sleep that I've put there, in reality it takes a different amount of time between each data point calculation, making the sine wave a bit wonky. This gets worse as more and more calculations are done in parallel
It's more likely to be the unaccuracy of Instant in rust. It's guaranteed to be non-decreasing, but not steady: https://doc.rust-lang.org/std/time/struct.Instant.html
> In other words, each tick of the underlying clock might not be the same length (e.g. some seconds may be longer than others). An instant may jump forwards or experience time dilation (slow down or speed up), but it will never go backwards
Looks like it's just plotting a fixed number of points at the time the iteration was scheduled. It's not trying to map a smooth curve.
I think it's just scheduling. The deformation is when it didn't get to run exactly in time because Tokio had to wake up enough other tasks first.
> Alright! We managed to plot two sine waves in the most convoluted method ever!
Those sine waves start to resemble a multi-voice music score. You might get some interesting pitch variability depending on other processes that are running.
This is not limited only to Rust and I'd be very curious to see a similar comparison across multiple runtimes!
Shared 12 days ago: https://news.ycombinator.com/item?id=43675098
The async disaster can be avoided by using Plan9 or Inferno.
Don't misinterpret this as criticism of async-rust. It's required to work on legacy platforms like linux.
This is obviously a troll comment, but it made me curious.
I'm familiar with Plan 9, and I know about the interesting mount namespace shenanigans you can pull off with it, but does it actually have anything that would help with async?
"Visualizing futures runtime like this really makes some pennies drop."
What's it mean to drop pennies?
It's a British idiom. When the penny drops you suddenly understand something.
https://en.wiktionary.org/wiki/the_penny_drops