Bananian

Slow Read and Write Speeds with SATA connection

21 4613
Edited by stevelambert at Sun Oct 4, 2015 15:38

I'm trying to troubleshoot slow read/write speeds on my Banana Pi connected to an external SATA drive. This BananaPi NAS is connected to my Asus N66U router via ethernet and I am connecting over wireless to the other machines. I realize there are many factors here, so I'm trying to eliminate a few.

I've tried transferring files from two other computers via SyncThing and rsync. Using rsync I get speeds that are around 7-10MB/s. With SyncThing it averages around 2MB/s.

My questions are these:

1. Is this a realistic speed, or should I expect more?
2. Given the details below, can I assume the BananaPi is set up correctly and start chasing after other points in the chain?


The Banana Pi is running the latest Bananian.

The drive is a 4TB drive with a gpt partition table and an ext4 filesystem.

I made sure it was connected as SATA2 like so:

  1. % dmesg | grep -i sata | grep 'link up'
  2. [    5.495912] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Copy the Code



Then I did some tests on the drive itself.

  1. % for i in 1 2 3; do sudo hdparm -tT /dev/sda; done

  2. /dev/sda:
  3. Timing cached reads:   584 MB in  2.00 seconds = 291.75 MB/sec
  4. Timing buffered disk reads: 200 MB in  3.06 seconds =  65.32 MB/sec

  5. /dev/sda:
  6. Timing cached reads:   674 MB in  2.00 seconds = 336.67 MB/sec
  7. Timing buffered disk reads: 262 MB in  3.00 seconds =  87.22 MB/sec

  8. /dev/sda:
  9.   Timing cached reads:   540 MB in  2.00 seconds = 269.73 MB/sec
  10. Timing buffered disk reads: 284 MB in  3.02 seconds =  94.08 MB/sec
Copy the Code

I also noticed when I copy a large file from one directory to another on the drive, the fastest speed I get is around 10MB/s.

Thanks in advance!

Verify that the disk isn't in NTFS. This for me resulted in data transfers like yours. I had to reformat to ext4. Went from 7MB/s to 30MB/s.

tkaiser  

pspcoelho replied at Mon Oct 5, 2015 08:57
Verify that the disk isn't in NTFS. This for me resulted in data transfers like yours. I had to refo ...

I checked this way and everything seems to be in order, right?

  1. sudo parted -l
  2. Model: ATA Mercury Elite Pr (scsi)
  3. Disk /dev/sda: 4001GB
  4. Sector size (logical/physical): 512B/512B
  5. Partition Table: gpt
  6. Disk Flags:

  7. Number  Start   End     Size    File system  Name     Flags
  8. 1      1049kB  4001GB  4001GB  ext4         primary  msftdata
Copy the Code
What would be the next step?


tkaiser replied at Thu Oct 8, 2015 07:03
http://linux-sunxi.org/Sunxi_devices_as_NAS#Benchmarking_.2F_Identifying_bottlenecks

I tried "iostat 5" as I was rsync'ing a file over the LAN - this was the result:

  1. Linux 3.4.108-bananian (bananapi)   10/12/2015  _armv7l (2 CPU)

  2. avg-cpu:  %user   %nice %system %iowait  %steal   %idle
  3.           15.61    0.00    1.93    0.14    0.00   82.31

  4. Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
  5. sda               2.13       191.92         0.05    1273337        336
  6. mmcblk0           1.73        21.72        48.75     144073     323424

  7. avg-cpu:  %user   %nice %system %iowait  %steal   %idle
  8.           45.38    0.00   30.63    1.45    0.00   22.53

  9. Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
  10. sda              79.40      9987.20         0.00      49936          0
  11. mmcblk0           0.20         0.00         0.80          0          4

  12. avg-cpu:  %user   %nice %system %iowait  %steal   %idle
  13.           45.42    0.00   29.87    1.54    0.00   23.17

  14. Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
  15. sda              77.80      9881.60         0.00      49408          0
  16. mmcblk0           0.20         0.00         2.40          0         12

  17. avg-cpu:  %user   %nice %system %iowait  %steal   %idle
  18.           47.88    0.00   30.26    0.10    0.00   21.76

  19. Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
  20. sda              81.20     10220.80         0.00      51104          0
  21. mmcblk0           0.40         0.00         1.60          0          8

  22. avg-cpu:  %user   %nice %system %iowait  %steal   %idle
  23.           47.42    0.00   28.47    0.81    0.00   23.30

  24. Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
  25. sda              79.60     10166.40         0.00      50832          0
  26. mmcblk0           0.20         0.00         0.80          0          4
Copy the Code

Anything look strange to you?

tkaiser  
stevelambert replied at Mon Oct 12, 2015 18:30
I tried "iostat 5" as I was rsync'ing a file over the LAN - this was the result:

And now?

If you do a combined test especially one that is CPU bound... what do you expect as a result?

Using rsync/scp or anything that's based on SSH for tests is always useless. Since the SSH ciphers are negotiated dynamically. So while between two hosts at location A a weak cipher will be used at location B a more strong will used so there the CPU becomes the bottleneck and comparing results isn't possible. And most likely using rsync with modern distros and slow machines like a Banana Pi/Pro the CPU becomes the bottleneck anyway (one CPU core completely busy doing encryption/decryption). This even applies to Xeon equipped machines. When we do server migrations we often rely on rsync (for incremental syncing) but we always use the weakest cipher possible (rsync -e 'ssh -c arcfour' -rltpog...) and for the initial syncs we have to setup many parallel rsync tasks since even Xeons that are a few years old aren't able to saturate half of normal GBit Ethernet speeds. If you've multiple GBit connections between the machines or 10 GbE you will end up with just 30-40 MB/s if you don't run more rsyncs in parallel. If this applies to 'big iron' servers it will also apply to 'tablet grade' hardware used on the Bananas.

This as well as the right testing methodology is explained in detail in the URL I posted. Test storage individually, test network individually and only if the results look promising start a combined test. Don't use tools that implement strong encryption to measure performance of network or storage. It simply does not work on slow ARM platforms when crypto acceleration can't be used (as it's the case with A20).

Edited by stevelambert at Wed Oct 14, 2015 06:01
tkaiser replied at Wed Oct 14, 2015 00:37
And now?

If you do a combined test especially one that is CPU bound... what do you expect as a re ...

Ok, fair enough. I'm just experienced enough to get myself lost here, so I appreciate the help.

I tried again with weaker encryption and got similar results in speed.

I set up Samba and tried that with similar results in speed, but far lower CPU numbers.

So, I do need to do some systematic testing as you described, however, I looked at the links you provided and I don't know where to start. I see this:
(local disk performance, network performance, combined network/disk performance)

and know I should start with local disk performance, but it's unclear what the next step is. What do I use to test disk performance - or was what I included in my initial post enough? And from there forward, how do I test network performance in a way that will get me the answers I'm looking for? Is there a tutorial somewhere that can walk me through this?

I tried to guess at how to proceed based on what I could understand and did this on the Banana Pi

  1. sudo netserver -p 22113
Copy the Code


and this on my desktop and got these results:

  1. netperf -H 192.168.1.60 -p 22113 -l 10
  2. MIGRATED TCP STREAM TEST from (null) (0.0.0.0) port 0 AF_INET to (null) () port 0 AF_INET
  3. Recv   Send    Send
  4. Socket Socket  Message  Elapsed
  5. Size   Size    Size     Time     Throughput
  6. bytes  bytes   bytes    secs.    10^6bits/sec

  7. 87380 131072 131072    10.14      63.93
Copy the Code

tkaiser  
Normally I use iperf and iozone (apt-get install iperf iozone3). I would also use them since you find many test results in the forums:

https://duckduckgo.com/?q=iperf+site%3Alemaker.org
https://duckduckgo.com/?q=iozone+site%3Alemaker.org

Reading through the results you will also stumble across tuning values but they depend on the kernel version you use and on other settings. So first try it without 'tuning'. Using iozone always use test file sizes at least twice as large as the amount of RAM available. If you don't have a second Linux, FreeBSD or OS X host around for iperf tests then I would use another tool (there exist many, eg. netperf -- they all share the behaviour to not mix network throughput with local I/O throughput which is important to identify the bottlenecks).

If possible you should test the different network 'zones' individually (Banana Pi <--> Router, Router <--> Client, Banana Pi <--> Client). Sometimes routing/bridging in home/SOHO routers is really slow and since you mentioned that your clients are connected through Wi-Fi I wouldn't even test since what can you expect using Wi-Fi? At least no stable or guaranteed minimal throughput since Wi-Fi means shared medium. The more devices use the same bands the less bandwidth available.

Edited by stevelambert at Wed Oct 14, 2015 09:24

Ok, did an iozone test (made a guess at the parameters to use).

  1.         Auto Mode
  2.         Using Minimum Record Size 512 kB
  3.         Using Maximum Record Size 1024 kB
  4.         File size set to 2048000 kB
  5.         Using maximum file size of 2048000 kilobytes.
  6.         Command line used: iozone -a -y 512k -q 1024K -s 2000m -g 2000m -i 0 -i1
  7.         Output is in kBytes/sec
  8.         Time Resolution = 0.000001 seconds.
  9.         Processor cache size set to 1024 kBytes.
  10.         Processor cache line size set to 32 bytes.
  11.         File stride size set to 17 * record size.
  12.                                                               random    random     bkwd    record    stride
  13.               kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
  14.          2048000     512    20082    20193    19330    19473
  15.          2048000    1024    19751    20162    19154    19568

  16. iozone test complete.
Copy the Code

Unfortunately, I don't know how to interpret the results.  Also, how do I know it's checking the SATA drive?

The iperf test will take me a little longer to figure out how to run on my router, but before I go on, does this throw any flags?

You have to log in before you can reply Login | Sign Up

Points Rules