Friday, November 27, 2015

Linux recovery of old photos and sticking them in cloud storage

So, the Western Digital My Book 1110 external drive we had for our local NAS is finally dying. It was connected to an old Buffalo router (WZR-HP-G300NH) and ran the not-so-glorious XFS filesystem. It would figure that some of the first sectors to go bad would be the ones containing the XFS superblocks. Sigh. Worse than that, the drive's S.M.A.R.T. statistics never reported an issue and smartctrl said everything was peachy keen, despite the read errors.

I looked into various recovery and cloud options. Being a Linux nerd and having a long Thanksgiving weekend, I looked into OneDrive, Google Drive, etc. Happily, I had purchased a new monitor for my wife and it came with a free year of Amazon Cloud storage. Furthermore, some enterprising geeks had made acd_cli which includes a FUSE driver for it so it mounts like other external drives in Linux (e.g., USB keys) and it looked better maintained than the OneDrive solutions that were available.

Great, so I have a place to store the data but how do I get it back? After giving up on fixing the filesystem — essentially unrecoverable according to the normal Linux XFS tools — I trolled forums looking for recovery software and came across UFS Explorer. Since it's shareware, you can download and try it out before buying. I downloaded it and it worked! It parsed the broken XFS partition without issue and now I could copy the files. SHUT UP AND TAKE MY MONEY. Due to the exchange rate to Euros and a 15% off promo code, it was extremely affordable and I bought it in a heartbeat.

Back to acd_cli. This was a pain to install. There were a lot of prerequisites to install for both compile and runtime. I haven't developed in python so the confusing python vs python3 thing was definitely a roadblock. After running 'make deb' in the assets directory, it failed. I ended up changing this line: 

@@ -1,5 +1,5 @@
 deb:
-       fpm -s python -t deb requests-toolbelt
+       fpm -s python -t deb --python-pip /usr/bin/pip3 --python-bin /usr/bin/python3 requests-toolbelt
        fpm -s python -t deb --python-pip /usr/bin/pip3 --python-bin /usr/bin/python3 ../setup.py


Finally, the debs built and I installed them. That was great but I had to add several obvious packages that were missing for runtime to function.

Eventually it worked after authorizing access, adding myself to the fuse group, and adding the --allow-root mount option so UFS Explorer (which runs as root) could see it.

acd_cli mount --allow-root ~/mnt

And now, after copying over 11000 files to the Amazon Cloud via this setup, I have come to realize that UFS Explorer really, really, really needs a mounting option for their recovery file systems. I can't use rsync or anything to pull the files directly into the cloud without an intermediary drive. I've noticed that some files copy over as zero length and I have to find them and manually recopy them. This makes me worry that some data could have been truncated-but-non-zero.

Whether this is the fault of acd_cli's beta quality lack of robustness or Amazon itself, I don't know. But command line tools like rsync and md5sum make this stuff easy despite flaky intermediary issues. Instead, I have doubts and few methods to assuage them other than brute force gui checking or buying yet another local drive, copying there, and then comparing to the cloud... which might still not be truly reflective of the state if UFS Explorer is flaky.

In conclusion, UFS Explorer is a cool tool and I appreciate the good quality of acd_cli and its aspirations. But, this is not a simple project for the faint of heart. You might consider a tool like UFS Explorer on Windows and using the Microsoft OneDrive sync tool on the desktop from an intermediary drive.

Monday, December 3, 2012

Fixing OpenAdmin Tool (OAT) for Informix on Ubuntu 12.10

After a long battle, I finally got OpenAdmin Tool (OAT) to work with Informix CSDK 3.70.FC6 on Ubuntu 12.10 64-bit. After symlinking all the libraries to fake the fact that I don't have the versions IBM wants (protip: use ldd on the php and apache binaries) and after installing OpenSSL 0.9.8, I still couldn't run StartApache. It kept failing with:

httpd: Syntax error on line 492 of $INFORMIXDIR/OAT/Apache_2.4.2/conf/httpd.conf: Cannot load $INFORMIXDIR/OAT/PHP_5.4.4/libphp5.so into server: $INFORMIXDIR/OAT/PHP_5.4.4/libphp5.so: undefined symbol: __dn_expand

The solution? Edit $INFORMIXDIR/OAT/StartApache

Change this line:

"$INFORMIXDIR/OAT/Apache_2.4.2"/bin/apachectl -f "$INFORMIXDIR/OAT/Apache_2.4.2/conf/httpd.conf" -k start

To this line:

LD_PRELOAD=/lib/x86_64-linux-gnu/libresolv.so.2 "$INFORMIXDIR/OAT/Apache_2.4.2"/bin/apachectl -f "$INFORMIXDIR/OAT/Apache_2.4.2/conf/httpd.conf" -k start

StartApache now runs successfully.

Monday, June 4, 2012

Switch to workspace 5, etc. with recent metacity

So, I had some trouble when I updated Debian sid (unstable) this morning. I could no longer switch to workspace 5 with my keybinding. Apparently, this is a result of the switch from gconf to gsettings. I always remap to ALT+F[1-5] because I hearken back to the fvwm and enlightenment days. You can just go to the normal "System Tools / Preferences / System Settings / Keyboard / Shortcuts / Navigation" menu to set switching to workspaces 1-4.

For workspace 5, though, you should go to "System Tools / dconf Editor". Navigate to "org.gnome.desktop.wm.keybindings". I just copy/pasted my setting for "switch-to-workspace-4" and changed the 4 to a 5. It is saved when you change it.

Thursday, May 31, 2012

Firefox Jumping to Other Workspaces in Linux

Random Ubuntu 12.04 note. If you are using Gnome Classic and Firefox and other windows keep switching ("jumping") to other workspaces, install compiz config and disable the "Place Windows" plugin (it's under Window Management).

Wednesday, April 11, 2012

MIT Fusion Researchers Update Us on Current Research

MIT Fusion Researchers Answer Your Questions

This is an awesome read! They really get into the nitty gritty of what fusion research has tackled over the last 10 years or so. Compliments of Slashdot.

Arrakis on Earth: The Caprivi Strip

When I imagine the world of Leto II in God Emperor of Dune, it looks something like these linear dunes of the Caprivi Strip.

"After the dunes formed, conditions in the Caprivi Strip moistened enough for the dunes to support vegetation — woodlands on the dune ridges, and grasses and shrubs in the valleys between."

It's just amazing to look at. Instead of regular hills and mountains carved by wind and water, you have sand dunes carved by wind alone and over the centuries, the build-up of moisture allows soil to form. From there, you get grass, trees, and the rest. But underneath it all is this history of sand and desert. Beautiful.