Discussion

Disapointingly slow disk writes of BananaPI

10 5806
When I first heard about a small board comparable to Raspberry but with Gbit LAN and SATA ports, I immediately was excited to use it as a small file server like WD MyCloud, which also has ARM hardware, but an awfully flawed firmware.

I hurried to buy two boards and started implementing the server using Debian.

However, today I must confess that I am disappointed because the writing speed to fast hard disks like WD red is only about 40 MBytes/s.
The disk itself on ext4 partitions located at outer parts of the platters can easily write more than 100 MB/s.

In other threads comparable speeds have been reported with guesses what might be the cause.

My analysis showed, that reading works at perfect speeds above 100 MB/s, so one can conclude that the DMA engine and RAM speeds for reading allow this range of speeds.

The inverse direction of data transfer imho should have about the same performance, and the only reason why it does not can only have two causes:

1) A bug in the silicone of the Allwinner chip degrades performance of the Hardware that gives good values when reading.

2) A bug in the kernel software 3.4.xx (eventially because the hardware definition file is bogeous) slows down disk write performance.

Imho it is the duty of the inventor and seller of the product - Lemaker Inc. - to fix that bug asap or to publish that it can not be fixed and that the writing speed for that chipset always will be limited.



I hope to soon get a competent answer from Lemaker.

Thanks and regards from Switzerland
LinAdmin

tkaiser  
You can expect 45/+200 MBytes/sec sequential speed (write/read). This is 'tablet/smartphone grade hardware' and I guess nobody at Allwinner (not Lemaker/SinoVoip/whoever-the-company-behind-Banana-is -- they simply put a SoC on a PCB) ever thought that people would be misusing this type of hardware for storage devices and the like

If you want to buy energy efficient ARM boards that feature both high network throughput and SATA speeds go with Marvell (and somewhat problematic linux support), if you also need GPIOs and the maker stuff the Minnowboard MAX is great. If you plan to store important data on such a solution ECC RAM is mandatory. If silence is a design goal I would then go with fanless AMD/Intel boards -- made good experiences with Avoton Atoms (ASRock C2750D4I, Gigabyte GA-9SISL or SuperMicro A1SAi-2750F)

tkaiser replied at Fri Nov 7, 2014 04:04
You can expect 45/+200 MBytes/sec sequential speed (write/read). This is 'tablet/smartphone grade ha ...

@tkaiser

I hope that you are wrong by saying that this A20 chip is hardware for handys and tablets only and never will perform better when writing to disk.

If indeed the hardware is limiting, you would have to explain why the bandwidth for SATA, DMA and RAM goes well above 100 MB/s for reading and stays badly low for writing, which from the standpoint of the silicone just reverses all directions?

I know that similar chips designed by TI and Freescale do not show this asymmetric anomaly and have above 100 MB/s in both directions!

I do not share your opinion that a small disk server requires 1 GB of EEC-RAM. I have studied the literature you mentioned and come to the conclusion that only servers with considerably more RAM chips should have EEC. And truly safe server operation can only be obtained when also the internal data paths of DMA etc. up to the CPU have hardware correction.

I still hope that Lemaker soon brings up a solution to this problem. My guess is that with probability of 80% it is a bug in kernel software (configuration) and 20% probability for a unintended hardware design bug which had not been detected before or which was detected by Allwinner but not published....

As every A20 platform I know (Olinuxino, CubieTruck, Banana Pi) has the same problem. I guess it should be a problem only Allwinner could solve. Judging from the difficulties of linux-sunxi group to get documentation / answers from Allwinner, I will not hope for a solution soon.

tkaiser  
Edited by tkaiser at Fri Nov 7, 2014 07:39

You should become a bit familiar with history/development of the linux-sunxi community (they are the people spending their spare time on your problems) and the history of the Allwinner SoCs (Android only in the beginning, their code known for many GPL violations in the past up until now, linux support started with Cubietech / Tom Cubie, a former Allwinner employee, if I'm not wrong). The Banana Pi is some sort of a late spin-off product of this process.

Regarding SATA speed I just said what you can expect today. If you would inform yourself a bit how the development process works regarding kernel/hardware stuff then you might consider asking politely the right questions in the right place and maybe donate some pieces of hardware so that volunteers have a suitable test setup when they work in their spare time on problems like this (maybe it's a simple driver issue, maybe not. The SATA section in the A20's officially released user manual contains just a few sentences). And you should consider telling kernel hackers how to measure storage correctly because these people tend to fail with such tasks (using dd or hdparm wrongly and therefore measuring always partially OS buffers/caches instead of storage devices).

Regarding ECC RAM it's a simple truth: If you don't have at least simple ECC memory (there exist 'advanced' ECC implementations for a reason: because simple ECC isn't enough since decades) you won't even be able to check whether you have faulty RAM or not (and if you have tons of servers and read out their ECC/EDAC error logs while running and check these machines with tools like memtest86 you will have to notice that there exist modules that survive 72 hours tortured with memtest86 without a single error and produce one correctable error every few weeks when in productive use. No ECC RAM --> no way to have a clue what's the health state of your memory )

With faulty RAM (non-ECC and ECC as well when we're speaking of 2-bit-errors) you might experience crashes (of programs or the whole system), you might experience silent data corruption (if this happens when writing file system structures then you might experience also huge data losses if your backup isn't existing) or you might not even notice anything because it either affects only data you will never access again (or where file formats are fault tolerant, interpolating missing pixels/frames) or everytime when these aforementioned things happen people want to believe it's just faulty software.

The reasons for bit flips do exist (even more in situations with unprotected boards like the Banana Pi -- compare with EMC), these sorts of problems increase with more and more electrons packed in fewer space (the LPDDR4 specs therefore optionally implement simple 'on die ECC' because these problems matter even on tablets/smartphones and the like), backups won't help that much because already faulty data will be backed up (this gets even worse with filesystems featuring end-to-end data integrity since in these situations you corrupt intact data on disk while scrubbing) but nobody likes to accept these facts

This complaint is pointless in my opinion.

1) You want to use your BananaPi as a small home server. OK.
2) You find out that SATA writes are slower than reads. OK.
3) You blame LeMaker for this. Well - this is slightly off since it's a problem of all Allwinner A20 based products.
4) But more importantly: Even if somebody would try to improve the SATA write performance, you would gain nothing with regards to your intended application!
You say you want to use this as a home server. The transfer speeds over Gbit ethernet on your BananaPi are not much faster than 40MB/s anyway! None of the A20 based boards come even close to the theoretical maximum network throughput of Gbit ethernet. So, you could increase the performance on your BananaPi locally but not over ethernet. And this is btw also seen on other boards like the i.MX6 based Cuboxes or Hummingboards where the Gbit ethernet port is limited to 480Mbps (yet, they put that in the specs and make it transparent).

That being said, Banana Pi will get mainline support in Kernel 3.19 (for basic headleass setup, no display/video support). A DMA driver is in the works, too (but most likely not to be included in 3.19).  So you might check how the performance of the BananaPi evolves with newer Kernel versions.

tkaiser replied at Fri Nov 7, 2014 06:48
You should become a bit familiar with history/development of the linux-sunxi community (they are the ...

Sorry I am not familiar with all details of the long term history of Allwinner and LeMaker.

To make it clear: I do appreciate the efforts of the sunxi guys, but it can not be their obligation to make available a kernel which allows buyers of the LeMaker products to realize applications for a board which they have paid to LeMaker.

So imho the correct business approach works as below:

A manufacturer offers a product with certain specifications and is obliged to deliver according to these specs.

When a manufacturer says that his wonderful board works with SATA 3 Gbit/s and says nothing more about real disk read and writing speeds, then I expect that data transfer happens at nominal 3 Gbit/s, of course limited by approx. 1 Gb/s by a WD red NAS disk.

Data transfer in both directions must happen at that speed.

Ethernet is limited to 1 Gbit/s and with the Realtek chip used on BananaPi I usually measure real transfer speeds of 0.9 Gbits/s on other hardware.

Conclusion: With a WD red disk and without design bugs in silicone or kernel software the BananaPi should transfer at about 800 Gbit/s over LAN, which happens to be 100 MBytes/s.

tkaiser  
Edited by tkaiser at Fri Nov 7, 2014 08:37
silentcreek replied at Fri Nov 7, 2014 08:04
The transfer speeds over Gbit ethernet on your BananaPi are not much faster than 40MB/s anyway! None of the A20 based boards come even close to the theoretical maximum network throughput of Gbit ethernet.


The 'funny' thing is: When doing 'real world benchmarking' (I collected some results here) you will be limited by the SATA limitation when it's about writing from a Client to Banana (iperf in this direction between 850/900 MBits/sec if settings are adjusted correctly: http://kaiser-edv.de/tmp/ooy7qm/iperf-results.txt). And in the other direction (Banana Pi writes from SATA disk through network to client) the network is the bottleneck -- see iperf results above.

In combination you get a NAS setup where you can write from client to server with max. 40 MBytes/sec and read with max. 60 MBytes/sec (and have full CPU utilization since everything's done 'in software'). Which is not that bad for such a tiny device never intended to become a 'server' (I still wonder why Allwinner included a SATA port in the first place? The older Mele TV boxes and the M5 use it and maybe some car entertainment solutions but in their main market it makes zero sense).

And as you already mentioned: On other platforms like eg. based on Freescales i.MX6 you get 85/100 MBytes/sec (write/read) SATA throughput which would perfectly match a GBit equipped board but the NIC's implementation is not just limited to 470 MBits/sec but at the time I played around with a WandBoard Quad last year the software support for the NIC was still very experimental and I experienced ultra low network speeds and RX errors as hell. Will give it a try again now that Igor has an image for another i.MX6 based boards.

silentcreek replied at Fri Nov 7, 2014 08:04
This complaint is pointless in my opinion.

1) You want to use your BananaPi as a small home server. ...

Re your points  intending to prove my arguments being pointless:

Ad 3) "... this is slightly off since it's a problem of all Allwinner A20 based products."

that is no proof because you do not analyze the situation in depth. It all depends if it is a bug in the silicone or in the currently available kernels.

Ad 4) I do not follow your argumentation, see my answer to tkaiser regarding Ethernet speed. If that hardware can not deliver according to the specs of the peripheral hardware (Ethernet and SATA), then the manufacturer has to make that clear in the specs!

My measurements of BananaPi Ethernet show, that transfer speed reading from disk and transmitting via Ethernet is much higher than what you make us believe!

I hope that soon a 3.16 kernel with good performance will be available. I am convinced that kernel version temporarily should be frozen at that version in order to become a part of Debian Jessie stable.

I still uphold my argument that it is the shop that sells boards has the obligation to make available a long term stable kernel with good performance. Otherwise it will remain a niche product of a few nerds and never become mainstream like Raspberry.

tkaiser  
Edited by tkaiser at Fri Nov 7, 2014 08:59
LinAdmin replied at Fri Nov 7, 2014 08:29
Ethernet is limited to 1 Gbit/s and with the Realtek chip used on BananaPi I usually measure real transfer speeds of 0.9 Gbits/s on other hardware.

Conclusion: With a WD red disk and without design bugs in silicone or kernel software the BananaPi should transfer at about 800 Gbit/s over LAN


No way. The RTL8211 acts just as PHY, the MAC part is inside the A20 (sunxi-gmac based on stmmac -- please keep in mind that this is 'tablet grade hardware' missing PCIe at all!), then there's nothing available like TCP offloading which might help free up the CPU and therefore results are as expected: You can nearly saturate GBit network speed (in one direction) and then the BananaPi is 0% idle since both CPU cores are busy doing low level network stuff. You can not compare the situation with a x86 box that can compensate crappy network hardware like Realtek's cheap designs due to CPU power.

And so on. It's not worth the time to further complain about poor performance since it is 'as expected'. When you already have your Banana Pis then set up monitoring (detailed instructions available here in the forum) and have a look at the load when you do either storage benchmarking or network benchmarking. And then think about the consequences when both will be used simultaneously. The CPU's missing power is the problem on a platform where everything has to be dealt with inside the CPU. This is low end hardware to play with and not 'server stuff'

The 'specs' (haha) claim that the Banana Pi is able to negotiate at GBit network link speed (which it does) and is able to negotiate with SATA disks at 3 Gbps (which it does). So where's the problem unless you're expecting miracles?

You have to log in before you can reply Login | Sign Up

Points Rules