I'm testing the hdd read write speeds when transferring a large media file across my local network. I have everything plugged into a gigabit switch. I'm using iotop to get the numbers. I have both Pogos setup to run the OS's off the attached hdd's.
The v3 Pro is set up with Debian:
The hdd is a 3.5" 500mb WD connected to the onboard sata connector.
hdd mounts bold:
The v4 mobile is running Arch.
The hdd is a Seagate 1tb slim+ usb3 capable, connected to the usb2 port.
hdd mounts bold:
For my initial testing, I used Thunar file browser in sftp mode. Thunar lets me connect to remote computers on my local network in a nice GUI environment to manipulate files. Unfortunately, using this method limited file transfer speeds to around 4 M/s for both read and write according to iotop. These results were pretty disappointing so I dug in to try to figure out why so slow. Turns out file transfers using ssh puts a lot of overhead on the cpu for encryption and there are some serious bottlenecks built into ssh itself.
A few small samples from the v4 using sftp:
Downloading
Uploading:
These next results were created after making a few changes. Switching to a less cpu intensive encryption algorithm, arcfour, and using the command line for file transfers using secure copy, scp. Debian allows arcfour whereas Arch required me to enable the use of it. This made a big difference, although the numbers still indicate pretty slow transfers. I think both Pogos are still being bottle necked by the encryption overhead. The v3 seems to have a slight edge in speed, possibly because it's dual core processor is more powerful than the v4's.
Used this to transfer file from my main x86 box to the pogo v3:
A sample of the results:
So based on what I have so far, it seems encryption overhead is still an issue. Switching to a weaker encryption algorithm has speed things up by nearly 3 times. How would eliminating it altogether effect speed? I spent some time trying to use a lower overhead method that would allow network file transfers with no encryption. Linux remote copy, or rcp would seem to be the perfect solution if not for being somewhat depreciated and having a lack of documentation on setup, at least that that I could find. I couldn't get it working in the end. I should also say rcp should only be used on a local network due to zero security built into using it.
There are many options available to move files around via network for Linux. I've intentionally avoided SMB Samba because I don't have any Windows box's and Samba seems like something band-aided to Linux for Windows.
With all that said, I will be using rsync for remote backups to one of the Pogos. It's slow but speed really doesn't matter for in this case, and after the first backup, only the changes will be sent.
A NAS box is a whole different situation, faster is better. What are you Linux users using for network file transfer protocol and any details?
The v3 Pro is set up with Debian:
[root@PogoV3oxnas ~]# uname -a Linux PogoV3oxnas 4.4.38-oxnas-tld-5 #1 SMP PREEMPT Sun Dec 11 17:32:48 PST 2016 armv6l GNU/Linux
The hdd is a 3.5" 500mb WD connected to the onboard sata connector.
[root@PogoV3oxnas ~]# lsblk -afm NAME FSTYPE LABEL MOUNTPOINT NAME SIZE OWNER GROUP MODE sda sda 465.8G root disk brw-rw---T |-sda1 ext3 rootfs / |-sda1 10G root disk brw-rw---T |-sda2 swap swap [SWAP] |-sda2 512M root disk brw-rw---T `-sda3 ext4 backup /hdd/sda3 `-sda3 455.3G root disk brw-rw---T loop0 loop0 root disk brw-rw---T loop1 loop1 root disk brw-rw---T loop2 loop2 root disk brw-rw---T loop3 loop3 root disk brw-rw---T loop4 loop4 root disk brw-rw---T loop5 loop5 root disk brw-rw---T loop6 loop6 root disk brw-rw---T loop7 loop7 root disk brw-rw---T mtdblock0 mtdblock0 14M root disk brw-rw---T mtdblock1 mtdblock1 114M root disk brw-rw---T
hdd mounts bold:
[root@PogoV3oxnas ~]# cat /proc/self/mountinfo 14 19 0:14 / /sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw 15 19 0:4 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw 16 19 0:6 / /dev rw,relatime - devtmpfs udev rw,size=10240k,nr_inodes=14726,mode=755 17 16 0:12 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620 18 19 0:15 / /run rw,nosuid,noexec,relatime - tmpfs tmpfs rw,size=12104k,mode=755 19 0 8:1 / / rw,relatime - ext3 /dev/disk/by-label/rootfs rw,data=ordered 20 18 0:16 / /run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k 21 18 0:17 / /run/shm rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=129060k 22 19 0:18 / /tmp rw,relatime - tmpfs tmpfs rw 23 19 8:3 / /hdd/sda3 rw,relatime - ext4 /dev/sda3 rw,data=ordered 24 19 0:19 / /var/lib/nfs/rpc_pipefs rw,relatime - rpc_pipefs rpc_pipefs rw
The v4 mobile is running Arch.
[root@JeffsPogo2 ~]# uname -a Linux JeffsPogo2 4.4.34-1-ARCH #1 PREEMPT Tue Nov 22 02:02:24 MST 2016 armv5tel GNU/Linux
The hdd is a Seagate 1tb slim+ usb3 capable, connected to the usb2 port.
[root@JeffsPogo2 ~]# lsblk -afm NAME FSTYPE LABEL UUID MOUNTPOINT NAME SIZE OWNER GROUP MODE sda sda 931.5G root disk brw-rw---- |-sda1 ext3 Seagate-Root b77d6532-1b83-4afc-a766-108ce4976de5 / |-sda1 4.4G root disk brw-rw---- |-sda2 swap swap c01a8152-20c4-410d-b143-dc1a659639c3 [SWAP] |-sda2 512M root disk brw-rw---- `-sda3 ext4 Seagate-Backup 969fdb6c-d4f1-4219-a605-e1c8d49fed29 /backup `-sda3 926.4G root disk brw-rw---- loop0 loop0 root disk brw-rw---- loop1 loop1 root disk brw-rw---- loop2 loop2 root disk brw-rw---- loop3 loop3 root disk brw-rw---- loop4 loop4 root disk brw-rw---- loop5 loop5 root disk brw-rw---- loop6 loop6 root disk brw-rw---- loop7 loop7 root disk brw-rw----
hdd mounts bold:
[root@JeffsPogo2 ~]# cat /proc/self/mountinfo 15 0 8:1 / / rw,relatime shared:1 - ext3 /dev/root rw,stripe=8191,data=ordered 16 15 0:6 / /dev rw,relatime shared:2 - devtmpfs devtmpfs rw,size=59848k,nr_inodes=14962,mode=755 17 15 0:15 / /sys rw,nosuid,nodev,noexec,relatime shared:5 - sysfs sysfs rw 18 15 0:4 / /proc rw,nosuid,nodev,noexec,relatime shared:9 - proc proc rw 19 17 0:16 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:6 - securityfs securityfs rw 20 16 0:17 / /dev/shm rw,nosuid,nodev shared:3 - tmpfs tmpfs rw 21 16 0:13 / /dev/pts rw,nosuid,noexec,relatime shared:4 - devpts devpts rw,gid=5,mode=620,ptmxmode=000 22 15 0:18 / /run rw,nosuid,nodev shared:10 - tmpfs tmpfs rw,mode=755 23 17 0:19 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:7 - tmpfs tmpfs ro,mode=755 24 23 0:20 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 25 23 0:21 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,freezer 26 23 0:22 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,cpuset 27 23 0:23 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,devices 28 23 0:24 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,blkio 29 23 0:25 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,net_cls,net_prio 30 23 0:26 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,perf_event 31 23 0:27 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:17 - cgroup cgroup rw,memory 32 23 0:28 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:18 - cgroup cgroup rw,cpu,cpuacct 33 18 0:29 / /proc/sys/fs/binfmt_misc rw,relatime shared:19 - autofs systemd-1 rw,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 34 17 0:7 / /sys/kernel/debug rw,relatime shared:20 - debugfs debugfs rw 35 17 0:30 / /sys/fs/fuse/connections rw,relatime shared:21 - fusectl fusectl rw 36 16 0:14 / /dev/mqueue rw,relatime shared:22 - mqueue mqueue rw 61 15 0:31 / /tmp rw,nosuid,nodev shared:23 - tmpfs tmpfs rw 60 15 8:3 / /backup rw,noatime shared:24 - ext4 /dev/sda3 rw,stripe=8191,data=ordered 175 22 0:34 / /run/user/0 rw,nosuid,nodev,relatime shared:133 - tmpfs tmpfs rw,size=11996k,mode=700
For my initial testing, I used Thunar file browser in sftp mode. Thunar lets me connect to remote computers on my local network in a nice GUI environment to manipulate files. Unfortunately, using this method limited file transfer speeds to around 4 M/s for both read and write according to iotop. These results were pretty disappointing so I dug in to try to figure out why so slow. Turns out file transfers using ssh puts a lot of overhead on the cpu for encryption and there are some serious bottlenecks built into ssh itself.
A few small samples from the v4 using sftp:
Downloading
526 be/4 jeff 0.00 B/s 2.38 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.54 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.43 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.44 M/s 0.00 % 0.10 % sftp-server 526 be/4 jeff 0.00 B/s 4.44 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.44 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.39 M/s 0.00 % 0.08 % sftp-server 526 be/4 jeff 0.00 B/s 4.31 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.34 M/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 0.00 B/s 4.26 M/s 0.00 % 3.38 % sftp-server
Uploading:
526 be/4 jeff 4.82 M/s 0.00 B/s 0.00 % 7.63 % sftp-server 526 be/4 jeff 4.12 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 4.14 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 4.14 M/s 0.00 B/s 0.00 % 0.25 % sftp-server 526 be/4 jeff 4.01 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 4.05 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 4.05 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 3.96 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 4.08 M/s 0.00 B/s 0.00 % 0.00 % sftp-server 526 be/4 jeff 4.22 M/s 0.00 B/s 0.00 % 0.00 % sftp-server
These next results were created after making a few changes. Switching to a less cpu intensive encryption algorithm, arcfour, and using the command line for file transfers using secure copy, scp. Debian allows arcfour whereas Arch required me to enable the use of it. This made a big difference, although the numbers still indicate pretty slow transfers. I think both Pogos are still being bottle necked by the encryption overhead. The v3 seems to have a slight edge in speed, possibly because it's dual core processor is more powerful than the v4's.
Used this to transfer file from my main x86 box to the pogo v3:
[jeff@Arch2014p9 ~]$ scp -C -c arcfour /mnt/1TB-WDHD/1TB-MOVIES/Boyhood.2014.mp4 root@192.168.2.92:/hdd/sda3
A sample of the results:
2845 be/4 root 0.00 B/s 8.78 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 8.81 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 9.40 M/s 0.00 % 0.08 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 9.64 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 11.60 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 9.95 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.39 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.74 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.36 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 11.25 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.96 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.81 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.56 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.86 M/s 0.00 % 0.63 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.81 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 11.17 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 10.21 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 9.99 M/s 0.00 % 0.07 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 9.91 M/s 0.00 % 0.00 % scp -t /hdd/sda3 2845 be/4 root 0.00 B/s 11.32 M/s 0.00 % 0.00 % scp -t /hdd/sda3
So based on what I have so far, it seems encryption overhead is still an issue. Switching to a weaker encryption algorithm has speed things up by nearly 3 times. How would eliminating it altogether effect speed? I spent some time trying to use a lower overhead method that would allow network file transfers with no encryption. Linux remote copy, or rcp would seem to be the perfect solution if not for being somewhat depreciated and having a lack of documentation on setup, at least that that I could find. I couldn't get it working in the end. I should also say rcp should only be used on a local network due to zero security built into using it.
There are many options available to move files around via network for Linux. I've intentionally avoided SMB Samba because I don't have any Windows box's and Samba seems like something band-aided to Linux for Windows.
With all that said, I will be using rsync for remote backups to one of the Pogos. It's slow but speed really doesn't matter for in this case, and after the first backup, only the changes will be sent.
A NAS box is a whole different situation, faster is better. What are you Linux users using for network file transfer protocol and any details?