4 years sitting in a desk drawer and now my Pine64 is back online.
A long time ago we did a review of a Kickstarter project known as the "Pine64." It was a $15 dollar arm processor board that came out after the Pi 2 in 2015. We were not one of the super-early backers and thus we got a 2016 board just like everyone else. When we did the review it wasn't overly favorable because during that era everyone was trying to "Kickstarter" their project but no one really had good foresight as to how to handle their product AFTER it was "kickstart' ed" for lack of a better word. We had hopes that in due time the OS would get more stable and these PIs that carry the Mali400 GPU chip sets would get useful again. So did it? Quick answer - Yes and No. Read on and if you want to know more.
The community of pine.
If there is one thing I have to give Pine64 massive Kudos for is the community itself. This is in part of multiple reasons. One of them is the fact that the company itself decided to move into the market of making a somewhat affordable laptop that really hits home with some of the tech people out there called the "PineBook." Another project this company is working on which didn't get at much traction but was unique to itself is the "PinePhone" which does not use an Android OS but instead, you get Linux on your phone. We shouldn't be surprised by this as the CEO of Pine used to work at Apple. So because they keep coming out with projects which are interesting, to say the least. The community has matured enough to be a stable ecosystem.
Unlike the OrangePi community which remains almost unchanged since my review with them; Or worse yet the NextThing C.H.I.P which instead of archiving their forums and just pulled the plug on their entire domain and vanished into the night. The Pine community has found its own base within the ARM platform which is a feat in itself.
The software of pine.
When you look at the list of operating systems that the Pine64 supports now versus what it used to support it's very impressive considering the fact that during our first initial review the website didn't really work properly and I had to rely on user forums to pull the correct images. But as we were diving into the majority of the operating systems we were finding a running theme with almost all of them.
Console only mode available with this image for Pine64 users.
Huh, rather than fight with getting the X-Desktop running many of the builds just focused on keeping the Pine64 in headless operation. Which by the way is great if you are running a website or a system service. But what if we wanted an X-Desktop experience for let's say... Capturing artwork on a website? It doesn't have to be fast! It just has to work! Well! To this end, we have to thank the Armbian community on this front because their image with just a few commands got our X-Desktop running with xfce4 working awesome. This style of the desktop is identical to what we had on the NextThing C.H.I.P. only this time around we have a 4-core 64-bit processor behind us instead of it a single-core 700mhz processor. So when you contrast things that way the answer to me is
Perhaps not desktop fast. But yes, it's faster! - S
Out of all of the operating system images I tried for Pine64 Armbian felt like the only one that tried to make a go at the Graphical desktop experience for the Pine64. If you wanted to talk about GPU acceleration or anything then you'd have to buy a PineBook as Mali400 support after 4 years is not really supported in any of the OS's out of the box. For those in the Pine community; Feel free to comment below if we're wrong. We would love to be wrong on this point because then I could load an OS with mali400 GPU support and play with Quake3 as we did on our Raspberry Pis! Or any other game that uses GLES acceleration!
Pine64 - Revisiting the hardware.
Since we're bringing this back from stasis from 4 years ago how about a few upgrades? For starters, we dropped it into a Hammond box to which we drilled mounting holes as well as ports on the side to connect everything up to all nice. And we also added the power and reset buttons which surprisingly now work within the Armbian OS that's super nice right there.
Finally, we ejected the crappy Chinese raspberry pi heat sinks in favor of a 38mm heat sink that was recycled off of an old IBM motherboard that was dead anyway. Unlike when we first got the Pine64 where there was no clock control of the CPU and thermal run away was always imminent. The newer OS like Armbian actually scales the processor back similar to how cel-phones handle the heat by not running at %100 all of the time. Understand also that most of those 3rd party heatsink kits for Raspberry Pi assume that you will also be running with active cooling. In which case yes the tiny heat sink actually does the job of keeping your processor cool. But a bigger heatsink means there's more surface area for the heat to dissipate.
The end result is this. With the Armbian OS loaded and CPU scaling from 600Mhz when idle and 1.5Ghz when at full power. The temps without a fan on the 3rd party heat-sink was 49C idle and 59C running. Effectively we are hitting the thermal throttle event point.
With the 38mm IBM north-bridge heat sink we were getting 25C when idle (Effectively room temperature) and 54C at full load. Also, keep in mind the time it took to reach 54C was over a course of 10 minutes whereas the 3rd party heat-sink got to the thermal throttle point in under a minute.
The only problem with this heat sink and it's more of a question mark than a problem is the memory and CPU share the same surface unlike before where it was just the processor on the same surface. It's unknown if it will generate problems down the road but for now, it's working great.
The last thing on my "TO-DO" list with this heat sink is to build a brace for it as we're using thermal grease between the heat sink and CPU and not double-sided tape like the third party heat-sink. We never liked double-sided thermal tape.
Rebuilding the software:
Okay, so as you already know we've switched to Armbian on our Pine64 and it's running well considering we do not have any 3-d acceleration going on in this unit. But usually when you fire up any computer you sort of need a purpose to do so. A reason. Otherwise, you're just collecting boards for the sake of collecting them. For the mid-2010's I was that guy until I got burned.
One project we had in mind for the Pine64 is to take over where the next thing C.H.I.P. was performing and eventually failed due to a bad eMMC chip that cannot be removed
The original NextThings C.H.I.P's job is to capture data in the form of GIFs and images on the Infochammel Chammelstrator which is an art board that people can draw on and is deployed on a system known as Teleborg that streams on both Twitch and YouTube.
This is how the process works:
- Setup a script to auto-launch Firefox.
- Login to Infochammel and/or a website that contains the same I-Frames as Infochammel to capture the data.
- Capture the data in PNG
- Point Firefox to a blank website after capturing a picture so that other people can draw on the artboard.
- wait 15 seconds and repeat the process all over again.
We had documentation to get xfce4 desktop running with a bunch of apt-get commands but really the only thing you need to do is go into:
armbian-config
Arrow down to system and press enter.
Install full firmware package and press enter for your pine64 to grab everything it needs for a desktop experience.
then you can back out.
Back at the main menu, you can go into software.
and from here you can enable desktop and have the Armbian auto-login.
Chammelstrator capture script on the Pine64.
Since we're at the xfce4 screen and we can see what's going on with another display this is when we began building our script out.
sudo mount -t cifs //123.123.123.123/share /home/localuser/chammelnet -o username=localuser,password='localpassword',uid=localuser,gid=localuser cls echo "Starting ChammelstationMiser 4270K" echo "Priming Firefox for Chammel injection" xdotool mousemove 1920 1080 xdg-open about:blank echo "Waiting 30 Seconds before the loop begins" sleep 30 xdotool key F11 echo "Loop begins now!" while : do xdotool mousemove 1920 1080 xdg-open https://website-with-html.com/chammel/ sleep 55 scrot -b -d 5 '/home/localuser/chammelpics/ChammelstrationMiser_4720K-MK2_%Y-%m-%d-%H:%M:%S.png' sleep 2 xdg-open about:blank sleep 900 done
We are assuming /home/localuser is dynamic and hope that you choose a better password than 'localpassword'.
Okay, recently I decided to SMB mount my Pine64 over to my NAS server. As we were going through a few SD cards and they were crashing rather hard after a few months of not only capturing data but compressing that data later. You don't have to do the "mount -t cifs" command. But instead, we would highly recommend transferring your work folder over to something that can take read/writes better like a USB stick or a SATA hard drive if your single-board computer can handle it like the Banana Pi M1.
Continuing onward; What this script does is it clears the screen with cls. Says a bunch of stuff with echo. Using an xdotool moves the mouse out of the way so we do not accidentally capture the mouse. xdg-open about:blank this tells our browser of choice in xfce4 which is Firefox-esr to open the about:blank page and show nothing. I wait 30 seconds just to make sure Firefox-esr is completely loaded before moving onward. Next, we xdotool key F11 to virtually press the F11 key forcing firefox to go into full screen. Now the loop begins.
With a "While:" we start an infinite loop to do. xdotool mousemove again to ensure that the mouse didn't drift back onto our screen and then xdg-open to a destination website. In our case, we made some custom HTML code with all of the IFrames that Infochammel uses for its chat room and for the artboard. As we are not concerned about making our website responsive we can cram both Iframes onto the same line since Infochammel's art board is approximately 720p. Then pause for 55 seconds to give the pine64 time to render the whiteboard system.
After this time we use a utility called scrot which is great for X11 screen captures. adding another -d 5 seconds delay; We take a picture and drop it into our user folder throwing a time-stamp on the end. Finally, go back to a blank screen to free up an artist slot and wait 15 minutes to loop again. Ta-da we have a script that endlessly takes pictures of an artboard random people draw on! For what purpose? To what end? Archival? Posterity? A really slow way to make an animation? You decide!
We were going to use ImageMagick for capture with the following command:
import -window root capture.png
but we found this to be slower than using scrot.
Battle of the VNC servers on the Pine64.
We needed to remote control this session so that we can check on it without having to turn on an LCD monitor.
We made a service script to bind x0vncserver which is a variant of TigetVNC that binds to screen 0 address. Now, if we were using this Pine64 as a desktop all of the time that would be perfectly fine. In our case, it's not really going to be connected to a monitor 90 percent of the time. I still want to plug in HDMI once and a while to do diagnostics but it was then I found out x0vncserver would not work for us. Because the moment HDMI gets unplugged the kernel cannot sense the display size and defaults to 1024x768 which is a real bummer.
Having screen:0 is great for testing. However, we found out later that we did not need to display a screen to use VNC. It was at this point we switched over to tightvncserver with the help of this documentation for OrangePi's but it worked just fine for Pine64 . but ran into additional problems.
xdotool woes.
We found that XDOtool and Tightvncserver do not play nicely with one another. Giving segmentation faults when we press our virtual keys. After a bit of research on GitHub we found that someone managed to write some code to patch this feature but it was surprisingly never merged into the main XDOtool line of GitHub. Afterward, it took a bit of digging as to how we can get the modifications but finally figured it out.
First, we had to get rid of XDOtool that came from the distro.
sudo apt-get remove xdotool.
now we can download the patch from GitHub. Take note that since we're now compiling software i needed to download some additional things because it threw some dependency errors. I probably could've gone into armbian-config again and downloaded the development kernel and header files. and be done with it. But once all 3-4 dependencies were satisfied it compiled without errors.
Also, a note with Debian Buster. For reasons beyond our ability to explain they took away pathing from the SU and the root account is Debian Buster. So I added it back in by
nano /root/.bashrc
and adding the following line.
PATH=/usr/local/sbin:/usr/sbin:/sbin:$PATH
So far, I see no adverse effects in restoring this outside of the fact that my programs can compile once again. Hopefully, in later versions, they'll fix that issue with Debian Buster. Other Pine64 operating systems may not face this issue.
wget https://github.com/jordansissel/xdotool/archive/edbbb7a8f664ceacbb2cffbe8ee4f5a26b5addc8.tar.gz ls xdotool-edbbb7a8f664ceacbb2cffbe8ee4f5a26b5addc8 make make install
Alright! Now XDOtool works with TightVNCserver. Fantastic! Thank you very much to the developer on GitHub for patching this for people who love TightVNCServer. You are a saint!
There is one last thing to wrap things up in a nice little bow here. Which is to make TightVNCServer launch with the channel account and to automatically load our script the moment the X-Desktop is generated. That way we don't have to log in to our TightVNCserver and restart the script in the event of a power outage.
For this I needed to generate a new script:
nano /usr/local/bin/tvncserver
and add the following:
#!/bin/bash
PATH="$PATH:/usr/bin/"
NAME="ChammelStrator"
DISPLAY="1"
DEPTH="24"
GEOMETRY="1920x1080"
OPTIONS="-name ${NAME} -depth ${DEPTH} -geometry ${GEOMETRY} :${DISPLAY}"
case "$1" in
start)
/usr/bin/tightvncserver ${OPTIONS}
sleep 2
DISPLAY=:1 xfce4-terminal --execute '/home/chammel/chammel' --title=Chammelstrator-Capture-12000k --hold &
;;
stop)
/usr/bin/tightvncserver -kill :${DISPLAY}
;;
restart)
$0 stop
$0 start
;;
esac
exit 0
make it executable.
chmod +x /usr/local/bin/tvncserver
Now we can stop/start/restart this like a service.
sudo /usr/local/bin/tvncserver start sudo /usr/local/bin/tvncserver stop sudo /usr/local/bin/tvncserver restart
The modifications from the original tutorial are as follows affecting /usr/local/bin/tvncserver:
- Replaced vncserver with tightvncserver since I have multiple vncservers on this pine64 i wanted to specify exactly what version we wish to run.
- added "DISPLAY=:1 xfce4-terminal --execute '/home/chammel/chammel' --title=Chammelstrator-Capture-12000k --hold &" with a two-second sleep timer so that the moment the X11 desktop for VNC is visible on Display=:1 to launch the chammel script stated earlier in this blog to begin capture.
Once tests are good I can continue with the tutorial and add it as a bootable service.
sudo nano /lib/systemd/system/tvncserver.service
and added the following.
[Unit] Description=Manage VNC Server on this droplet [Service] Type=forking ExecStart=/usr/local/bin/tvncserver start ExecStop=/usr/local/bin/tvncserver stop ExecReload=/usr/local/bin/tvncserver restart User=chammel [Install] WantedBy=multi-user.target
This allows the script to be launched by the designated "chammel" account we made on the Pine 64.
sudo systemctl daemon-reload sudo systemctl enable tncserver.service
Now we enable this new service on our Pine64 and restart and our script takes off running all by itself. We can also access this by command line in case we need to do any maintenance to our Pine64.
sudo systemctl start tncserver.service sudo systemctl stop tncserver.service sudo systemctl restart tncserver.service sudo systemctl status tncserver.service
The tutorial goes on about how to use VNC through SSH but in our scenario, since the Pine64 is within our network is firewalled away I don't have to worry about it being too great of a security risk.
Preventative maintenance of our Pine64.
Since this entire automation system is made by the lowest bidder and the CPU is hardly being taxed we could do some things to further optimize the data that we receive from Infochammels servers.
For starters we can optimize some PNGs.
sudo apt-get install optipng
nano /etc/cron.hourly/optimize-pngs
And this is what we put into this file.
#!/bin/sh PATH=/sbin:/usr/sbin/:/usr/local/sbin:/bin:/usr/local/bin
mv /home/chammel/chammelpic/* /home/chammel/chammelproc/
nice -n 19 find /home/chammel/chammelproc -type f -name "*.png" -exec /usr/bin/optipng -o4 -strip all {} \;
mv /home/chammel/chammelproc/* /home/chammel/chammeldone/
chown chammel:chammel /home/chammel/chammeldone/*
Okay, so we made three directories.
- chammelpic is where scrot drops the screen captures into.
- chammelproc is the processing area where we run optipng. Placing a nice value of 19 so it's the lowest processor priority in the event the chammelmiser is taking a picture.
- chammeldone is the final PNG file that can be archived.
Why three directories? It's so that we can optimize only the PNGs we received instead of cycling through every single PNG file. Over the course of months of operation, there's going be thousands of these files.
What's the advantage of optipng? As described in a previous blog many programs that save in PNG either don't optimize or put too much meta-tag information into the file. The program "scrot" is no different. After we process these files we can save on average 20-30 percent over the original without altering the image at all.
Back up that sweet sweet Pine64 data!
nano /etc/cron.daily/7zip
We're going to be using probably the most aggressive archival utility out there 7-zip. As we do not care about the permissions of the files.
#!/bin/sh PATH=/sbin:/usr/sbin/:/usr/local/sbin:/bin:/usr/local/bin rm /home/chammel/ChammelstrationMiser_4720K-MK2.7z nice -n 19 7z a -mmt2 /home/chammel/ChammelstrationMiser_4720K-MK2.7z /home/chammel/chammeldone chown chammel:chammel /home/chammel/ChammelstrationMiser_4720K-MK2.7z
Running this once a day presents us with a 7-zip archive that compresses our files further as well as makes the files easy to download or pscp transfer over to an offsite server in the event the SD-Card on the Pine64 or perhaps the Pine64 dies a horrid death.
Now! Interestingly, even with using a Nice value of 19 if we left 7-zip to its own devices, it would lock up the pine64 hard by consuming all processor cycles to making the archive. Which is why we restrict the number of processors it will take to 2 with the -mmt2 command.
The end result?
Instead of me staring at an "about:blank" screen I have my script pointed to an area of this server that's used for scripting and experimentation to which this screen shows up when you access that part of the website. I kind of want my pine64 to perpetually go out to the internet because if it has no internet then the whole Infochammel capture system isn't going to work anyways. I had to alt+tab to show you the script window in the background which it's echoing ls -l /home/chammel/chammelpic to verify if there are any pictures in there which currently there is none. Otherwise, the script is working perfectly fine and capturing all of the artwork's goodness.
Unresponsive script warnings!
ChammelstrationMiser 4720K-MK2 almost went spiraling out of control when we were encountering unresponsive script warnings. Because the script could not detect this pop-up it just kept making more and more and more tabs until eventually Firefox simply crashed.
Similar in the article on how we suppressed the tabs from showing in full-screen. We were able to go to the about:config section of firefox once again and find dom.max_script_run_time and change it to "0" as anything that is zero or a negative value number is disabled. we would rather have our script simply give up and walk away then throw pop-ups we can't close while the automated system runs.
having additional tabs open was also keeping Chammelstrator from going full screen and thus making the capture less optimal with having the desktop in the background. And having the Firefox crash report tab popping open is no good! Checking browser.sessionstore.resume_from_crash to false is another option we had to set in Firefox's configuration.
Back up your Pine64!
Backups are important. After all, we're running on technology that has been produced by the lowest bidder using PNY memory cards from wal-mart for $3. Not that we're complaining about the cheapness of data at all. But it's always good to back up the data you have collected and played it safe in case of total failure.
Of course, we could use win32 Disk Imager and backup all of the work we put into the Pine64 which we have done. But what about all of the data that's been collected throughout the days/months/years? well!
sudo nano /etc/cron.daily/compress-chammel
Then add the following:
#!/bin/sh rm /home/chammel/ChammelstrationMiser_4720K-MK2.7z /usr/bin/nice -n 19 /usr/bin/7z a -mmt2 /home/chammel/ChammelstrationMiser_4720K-MK2.7z /home/chammel/chammeldone chown chammel:chammel /home/chammel/ChammelstrationMiser_4720K-MK2.7z /usr/bin/pscp -sftp -r -i /special/directory/for/keys/.key/id_rsa.ppk -pw 'Sup@S3cr3+P@ssw0rd!?!' /home/chammel/ChammelstrationMiser_4720K-MK2.7z randouser@server.com:/chammel
Alright, since we have some processing power to spare with the Pine64's cores we decided to 7-zip the snapshots of our backed-up data once a day and upload it to the webserver that we use online. Now, we did find it to be important to use single quotes for the password in case you're like us and use characters, capitalizations, numbers, etc within your password as you should. Don't use weak passwords kids. This process tends to take about 5 minutes right now but we know time will increase as the size of this file gets bigger. Also, we probably should be doing all of this on a USB-Thumb Drive instead of the Pine64's internal SD card but at $3 we're not terribly worried about it. Since we not only made an image of the OS but also backed up all of the data to the remote server data loss should be minimal.
If you want to use a ppk key to make your SSH a little more secure check out this article.
Finally, on the other side of "server.com" I log in to that and set up another cron.daily job to push it up to this very website.
Download that data!
Since you've made it to the end of this blog you probably want some proof of what we're doing so that we're not bullshitting you. I've made the ChammelstrationMiser_4720K-MK2.7z downloadable for everyone seeing this. It contains images from Infochammels - Chammelstrator artboard as well as a snap-shot of the chat at the time as well. perhaps more info will be added later on but for now, it's pretty sweet. Perhaps in the future after a year's worth of captured data; We make a compilation video. who knows!
The archive is updated every 24 hours automatically. So if you want an update just smash that download button again after 24 hours.
Afterthoughts.
Well, that's a waste, what else does your pine do?
You'd be right if that's the ONLY thing we're using the Pine64 for. after-all it has 4 cores of processing goodness right? Because this box is going to be running 24/7 collecting and capturing data minus will put it to good use and help out my network while we're at it.
Enter the Pi-Hole.
This utility on Linux allows you to set up a DNS cache server with a twist. It also filters out any advertisement servers from your network. So instead of pointing your DNS out to google (8.8.8.8 or 4.4.4.4) you point it instead to Pi-hole and because it's a DNS cache it speeds up your network. If a website does force you over to an ad-server you will simply be met with a 404 error instead of downloading any of their garbage on your PC.
curl -sSL https://install.pi-hole.net | bash
Even though the entire Linux community will jump up and down screaming that you should never-ever pipe curl into bash. Especially while logged into the root. It's a way of making the entire step of launching the installation utility with one command. I had to correct some dependencies that Raspberry Pi had whereas the Pine64 did not. but otherwise, the installation went smoothly. I'm probably going to redo my BananaPi in the future and also add a pi-hole to that. That way, if one of my ARM servers crashes it does not take down my whole network.
Final thoughts.
In the future, we may dedicate some CPU cycles to folding @ home or maybe some crypto work. Also, get this Pine64 out of a black Hammond bomb aesthetic case and into something a little more pretty like what we did for my Raspberry Pi 3. But ultimately we're finding a use for our $15 dollar board which is all reality that's what counts as a 'win' in the ARM processor business. We do hope that someone figures out how to use the Mali400 chipset so we can see the full potential of the Pine64. But for now, this is more than acceptable. This isn't a tutorial as much as it was a place where we dumped my notes on how to get the Pine64 working in the event it crashes. If it helps you guys out in a project similar then that's cool too.
If you've gotten this far we thank you for reading this and until next time -
Server protect you.
+++END OF LINE
Thanks a lot! Great post!
Thanks a lot man!