Is Amazon Lightsail worth it?

Amazon AWS Lightsail review 2018

In November of 2016 AWS launched its brand Amazon Lightsail to target the ever growing market that DigitalOcean, Linode and co. made popular. While AWS launched Lightsail as a complete seperate website back then, they recently decided to integrate it in the regular AWS website and after DigitalOcean and Linode adjusted its pricing they did so as well.

I personally think competition in this space is always great, because in the end it simply means cheaper services for us customers. This is why on paper Lightsail looks great and I actually like the idea behind it to bring new customers on board with a simple interface similar to DigitalOcean. Because of the recent changes I looked into Lightsail once more to see how those machines are actually performing and if they are worth it for my projects.

Apart from the change in price they seem to continously bring new features to Lightsail, even though they are already present on the AWS platform. For example the launch of their DBaaS service on the Lightsail platform. In this review I will focus only on compute, plans there start at $3,50 for which you will get 512 MB RAM, 1 CPU thread and 20 GB SSD disk. The $5 plan will give you access to 1 GB RAM, 1 CPU thread and 40 GB SSD disk.

What I noticed right away when launching some Lightsail instances is that they are heavily throttled, so while the pricing sounds competitive at first, you might be surprised by the performance when actually using Lightsail for anything serious that requires the resources. This is why I decided to write a short review of Amazon Lightsail and show some basic benchmarks.

How your cloud instances are being throttled

Before going into the actual Amazon Lightsail benchmarks I would like to explain what I mean by throttling. By using virtual machines, one needs to understand that basically everything can be shared on the host system (the underlying hardware). While it’s a bit harder for Memory, the CPU power is shared, Disks can be oversold and you share the underlying physical disks. you probably also don’t need your traffic quota anyways and so on.

A typical host system in the “cloud world” will have a Dual CPU setup with at least 384 GB of RAM. The disks, by default, are either attached over the network (Amazon Lightsail) or are local storage (DigitalOcean, Linode, etc.). Both have Pros and Cons, which is another topic but in the first case that means that actual network bandwidth becomes a factor as well, since you need it to connect to your disks, so there is not only an actual bottle neck by the disks and its interface but also network congestion might become an issue down the road.

If we do some basic calculations and assume that 380 GB of the 384 GB are given to customers and the host system has 48 CPU threads (2 CPUs with 12 cores each) that means that every customer has access to 0,13 threads. So if you burst your single thread all the time, you are actually stealing calculated CPU time from more than 10 other customers. Also a thread is usually not nearly as strong as a real core, so the situation is even worse.

Usually not every customer is using 100% of their CPU power so it automatically balances itself out fairly well. But I just wanted to demonstrate that only because the pricing page says you have access to 1 CPU “core” does not mean that it is actually your core freely to use all the time.

This is why some providers limit or throttle resources before hand, so that you will never experience any problems with bad neighbours, which is the case for Amazon Lightsail. This has Pros and Cons as well but I think you need to understand that simply comparing specs and thinking that Lightsail is very very competitive might be not a smart thing to do.

What exactly is under the hood?

When looking at specs and performance we should understand how good or strong the underlying hardware actually is. This is why I spawned several instances in different regions and availability zones to see how it compares and if there is any difference for the different plans.

CPU Models used by Amazon Lightsail(tested in multiple regions and availability zones)

Most of the time the virtual machines had the following CPU:
Intel® Xeon® CPU E5-2676 v3 @ 2.40GHz

With bigger instances I sometimes, rarely, got:
Intel® Xeon® CPU E5-2686 v4 @ 2.30GHz

Both are custom CPUs Intel is manufacturing in cooperation with AWS. The first seems to have 12 cores and 24 threads while the second one got 16 cores with 32 threads. Which shows that my calculation from above seems to be not really far off and seems to be realistic for Amazon Lightsail as well.

Amazon Lightsail CPU performance

Actual CPU performance is easy to benchmark, the problem in the cloud is that sustained CPU usage is a problem for other customers. This is why most providers worked on some intelligent systems that throttle your assigned CPU threads after some time of bursting. The question is not if this happens but when, so I wrote a small script that simply uses 100% CPU resources for several hours and detects when the CPU performance is declining rapidly. This tells us that the provider is limiting or throttling resources for this single virtual machine.

Plan Threads Sysbench (ST) Sysbench (MT) Time until throttling Aft. thro.
512 MB 1 29.34 s ∼ 30 min. (sysb. 583.12 s) 5 %
1 GB 1 30.38 s ∼ 30 min. (sysb. 311.99 s) 10 %
2 GB 1 29.99 s ∼ 45 min. (sysb. 96.14 s) 30 %
4 GB 2 27.91 s 13.92 s ∼ 30 min. (sysb. 71.21 s) 30 %
8 GB 2 27.94 s 14.01 s ∼ 45 min. (sysb. 47.29 s) 45 %
16 GB 4 29.83 s 7.44 s ∼ 40 min. (sysb. 88.05 s) 35 %
32 GB 8 28.01 s 3.49 s ∼ 30 min. (sysb. 113.89 s) 25 %

Sysbench was running until throttling kicked in and the average of all those values before throttling were taken. The results are very similar to each other and also the multi thread benchmark shows the expected result. ST stands for single thread, while the time at MT shows multi threaded performance. Lower is better.

What we can see is that while the CPU performance is good in the beginning, those threads are being throttled very fast. After throttling the performance is simply not there. For example, with the first plan you can burst around 30 minutes and after that the same benchmark that took around 30 seconds before, was taking 583 seconds after throttling kicked in.

What I noticed was that for example while the three cheapest plans all have one single thread, the throttle performance is different. After throttling kicks in the 512 MB plan can only burst up to 5 % of that one thread, the 1 GB plan 10 % and the 2 GB plan up to 30 %. I have added this value to the table at "Aft. Thro.". For plans with multiple threads it seems that one thread is throttled by the percentage shown and the other one is disabled during the "throttling phase", but there might also be something else going on which I wasn't able to identify.

In general, for any serious workload this might cause problems down the road, think about when your database queries suddenly take around 15x-20x longer (or even more) without you really knowing why.

Amazon Lightsail Disk performance

When testing disks there are several things to look at, nowadays there might be huge differences between "SSD storage" and "SSD storage". The focus on NVMe SSDs the last years is also swapping over to the server world, where NVMe SSDs are becoming more and more common also for cloud providers. Aside from SSDs vs. NVMe SSDs it's also important if the disks are attached directly to the server or through the network. Besides the regular disk bandwidth and IOPS I also looked into disk latency to see if there are any bottlenecks with Lightsail.

Before throttling

Plan Read KB/s Write KB/s Read IOPS Write IOPS Latency
512 MB 12,384 12,372 3,096 3,095 347 μs
1 GB 12,310 12,298 3,077 3,074 371 μs
2 GB 17,534 12,229 4,383 3,057 317 μs
4 GB 33,786 12,184 8,446 3,045 299 μs
8 GB 70,904 11,836 17,725 2,958 292 μs
16 GB 140,485 11,929 35,121 2,982 319 μs
32 GB 254,842 11,712 63,710 2,927 369 μs

After throttling kicks in

Plan Time u. throt. Read KB/s Write KB/s Read IOPS Write IOPS
512 MB ∼ 30 min. 414 (∼ 3%) 409 (∼ 3%) 100 (∼ 3%) 100 (∼ 3%)
1 GB ∼ 30 min. 505 (∼ 4%) 536 (∼ 4%) 120 (∼ 4%) 120 (∼ 4%)
2 GB ∼ 30 min. 745 (∼ 4%) 720 (∼ 6%) 180 (∼ 4%) 180 (∼ 6%)
4 GB ∼ 30 min. 987 (∼ 3%) 988 (∼ 8%) 240 (∼ 3%) 240 (∼ 8%)
8 GB ∼ 30 min. 1,938 (∼ 3%) 1,988 (∼ 16%) 480 (∼ 3%) 480 (∼ 16%)
16 GB ∼ 45 min. 3,887 (∼ 3%) 3,903 (∼ 33%) 960 (∼ 3%) 960 (∼ 32%)
32 GB ∼ 90 min. 7,748 (∼ 3%) 7,830 (∼ 66%) 1920 (∼ 3%) 1920 (∼ 66%)

In general we can see that the read performance quota is getting better throughout the plans, so upgrading from one plan to another would increase read performance significantly. Write performance on the other hand is equal among all plans, so if you have a heavy write use-case don't bother upgrading your plan. Apart from that that I think the before-throttling values are fair, even though not super good. What you don't see from the average latency is that I noticed some spikes to 2-3 ms now and than, so it's not consistently in the μs range.

Throttling kicks in after about 30 minutes for all plans except the two most expensive ones. After throttling the disks are crippled and in my opinion not really usable for any serious workload. As you can see for disk reads it's about 3% of the original values and write values increase the better the plan gets, because writes are already throttled from the beginning, otherwise everything would be in the 3% range.

The latency varies but seems to be consistently within the 300-400 range on average. I wouldn't read too much into the actual values and won't compare them among the plans, because some had spikes to some ms and so the average value for some plans a bit higher. Latency in general seems to be good, after seeing those values I was actually not really sure if Lightsail is also block based (as thought previously) or local storage.

This is why I spawned another instance and attached a volume to it. I also performed the same latency test on that external volume and it shows the same results as the default disk, which means that Lightsail Instances are completely block based. By doing so I also noticed that it shows the default disk is block based in their web panel, but I always like to double check things myself and by doing so I now know that this is true ;-).

Amazon Lightsail Network performance

The good thing about Lightsail is that there is, depending on the plan, some TBs of traffic already included. Additional traffic is the same price as with regular AWS instances though, so going over the included limit will be very expensive. The question I asked myself beforehand was if the network quality is the same as with regular instances or if they use some separated budget network for Lightsail. From what I can tell it is exactly the same, that means that direct connections to expensive providers like DTAG are also available with Lightsail.

Plan Inbound Outbound Time until throttling After throttling
512 MB 500 Mbps 500 Mbps ∼ 5 min. 30 Mbps
1 GB 1 Gbps 1 Gbps ∼ 5 min. 60 Mbps
2 GB 1 Gbps 1 Gbps ∼ 5 min. 125 Mbps
4 GB 1 Gbps 1 Gbps ∼ 5 min. 250 Mbps
8 GB 1 Gbps 1 Gbps ∼ 5 min. 500 Mbps
16 GB 1 Gbps 1 Gbps ∼ 5 min. 750 Mbps
32 GB 1 Gbps 1 Gbps

Before the throttling kicks in (after around 5 minutes), there is always enough bandwidth available. For the smallest plan it's possible to burst 500 Mbps in and out, while all other plans have access to 1 Gbps. Compared to the competition the 1 Gbps is on the lower end but still more than enough for most use-cases. The degree of throttling depends on the plan, but for example with the $5 plan after throttling you only have access to 60 Mbps.

Conclusion

The idea behind Lightsail is great, I also like that they put an effort into UI/UX. It shows! The interface is simple, easy and makes it a joy to use – basically the complete opposite of the regular AWS interface. I also like their offerings and that they are priced competitively. The small things like being able to add up to five IP addresses per instance for free or that basic DNS zones are included is great. It's a change from the regular AWS pricing, where you can bet that even the most simple requests or queries will be charged accordingly.

What I dislike is that everything is throttled. It's a nightmare to deal with, because it's hidden and not really clear when it happens. If you simply look at the pricing page there is no indicator that instances are being throttled that heavily, so it would be nice to know about that before hand or even know how much CPU credits one has used already and how much power is available per instance. Of course I don't expect to be able to burst the CPU performance all the time, but when I am in need of some performance I don't want my provider to shut me down.

For me personally, even though I really want to like it, there is no use-case for Lightsail, which is why I won't use it for my projects. I simply don't trust that throttling system in its current state for any productive workloads and have no need for expensive development machines. If you have anything non public facing that will be used sometimes throughout the day, like a Gogs instance, I think Lightsail is great. Other than that, even for a simple, unoptimized, WordPress blog, it's risky. One blog post going viral means you will outgrow your instances very fast and risk throttling even on the biggest machines, so it might not even be an option to upgrade to sustain load.

Feedback and Suggestions

I always appreciate feedback and suggestions. You can contact me directly via niklas [at] karoly [dot] io.