I just noticed the Coisas CMS (https://github.com/fiatjaf/coisas) for editing Markdown etc content on Github. So cool!
TODO: Link a lot of my prior work/research into this area
How do I save?
Keyboard shortcuts?
How is it on Mobile?
My phone running Android stopped being able to read its Adoptable Storage™ and so lost all non-stock apps and their data (Riot, OSMand+, Signal, etc) as well as the shared “home dir” containing DCIM, Documents, Downloads. Most of this is backed up in various ways but I’d like to get the whole storage back.
The phone is rooted, using the LineageOS method with ADB root access enabled, so I can copy the encryption keys with a USB cable.
Solid state memory devices are sensitive to power supply issues, and I’ve long suspected my phone’s power supply is flakey since I often see spontaneous reboots and assorted random failures. My hope is that the card will be readable by a device with better power (it was!)
Note: 1 and 4 can be completed without 2 and 3, although resizing the partition to the new card size wouldn’t be possible.
I removed the SD card from the phone and inserted it into my Linux laptop’s SD port.
First rule of storage device recovery: create an image and work on the image.
Note: I’m creating an image of the entire device rather than just the encrypted partition. This will allow me to restore to a new card later.
dd, but saw I/O errors.
dd~/backups/herolte_recovery
❯ sudo dd if=/dev/mmcblk0 of=herolte_sd_recovery_fulldisk.img status=progress
19326185+0 records in
19326184+0 records out
9895006208 bytes (9.9 GB, 9.2 GiB) copied, 208.766 s, 47.4 MB/s
dd: error reading '/dev/mmcblk0': Input/output error
Indeed,
❯ journalctl -b --since "10 minutes ago"
Mar 24 13:13:52 chip kernel: print_req_error: I/O error, dev mmcblk0, sector 19359824 flags 80700
Mar 24 13:13:52 chip kernel: print_req_error: I/O error, dev mmcblk0, sector 19359871 flags 80700
<30 lines omitted>
Mar 24 13:15:23 chip kernel: print_req_error: I/O error, dev mmcblk0, sector 22323535 flags 0
Mar 24 13:15:23 chip kernel: Buffer I/O error on dev mmcblk0, logical block 22323535, async page read
Time to try ddrescue, a time-tested utility for reading data from dying block devices. (archwiki)
❯ sudo ddrescue -f -n /dev/mmcblk0 herolte_sd_recovery_fulldisk.img herolte_sd_recovery_fulldisk.log
GNU ddrescue 1.23
Press Ctrl-C to interrupt
ipos: 31673 MB, non-trimmed: 0 B, current rate: 9092 kB/s
opos: 31673 MB, non-scraped: 0 B, average rate: 32025 kB/s
non-tried: 0 B, bad-sector: 0 B, error rate: 0 B/s
rescued: 31673 MB, bad areas: 0, run time: 16m 28s
pct rescued: 100.00%, read errors: 0, remaining time: n/a
time since last successful read: n/a
Finished
That went considerably better, no IO errors at all. Why the difference from dd? ddrescue is optimized in various ways to deal with unreliable block devices.
@kapitanpetko’s post from 2015 is the original source all the related stack exchange articles seem to draw from (usually without reference).
Allow reading naughty bits (requires phone rooted for adb access):
❯ adb root
* daemon not running; starting now at tcp:5037
* daemon started successfully
restarting adbd as root
Fetch the key:
❯ adb pull /data/misc/vold
/data/misc/vold/: 1 file pulled. 0.0 MB/s (16 bytes in 0.013s)
❯ ls vold
bench expand_ffffffffffffffffffffffffffffffff.key
# convert bin to hex
❯ hexdump -e '16/1 "%02x" "\n"' vold/expand_ffffffffffffffffffffffffffffffff.key
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa%
(source for plain hexdump format string)
First map the image to a loop device so dmsetup can use it. Note that the -P flag creates a device for each partition in the image.
❯ sudo losetup --show --find -P herolte_sd_recovery_fulldisk.img
/dev/loop0
chip# dmsetup create crypt1 --table "0 `blockdev --getsize /dev/loop0p2` crypt aes-cbc-essiv:sha256 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 /dev/loop0p2 0"Mount filesystem:
❯ mkdir mnt
❯ sudo mount /dev/mapper/crypt1 mnt
❯ ls mnt
app local lost+found media misc user user_de
Looks good!
Side note: Option 2 was a dead end because Android didn’t see any of the apps.
The 32GB replacement cards I ordered are actually physically smaller (28.5G vs 29.5G) than the original so I wouldn’t be able to restore a full device image to one of them. I have a 64G card that I could copy to the image to but that would likely result in half the card being unused, so I opted to try copying the data to a new encrypted partion.
rsync -a srcmnt/ destmnt.Unfortunately as noted above, android doesn’t recognized apps copied over this way.
Plan:
dd the image to new card.Ok, now what I’d like to do is copy the entire filesystem to a new card. My new card is 2x the size of the old card, so I’ll grow the partition after copying. If your card is smaller you’ll need to shrink the partition before copying.
❯ sudo dd of=/dev/mmcblk0 if=herolte_sd_recovery_fulldisk.img bs=4M status=progress
[sudo] password for eric:
7555+1 records in
7555+1 records out
31691112448 bytes (32 GB, 30 GiB) copied, 729.118 s, 43.5 MB/s
At this point I ran sync and put the card back in the phone. It works!
The only problem is, because I copied a 32G image to a 64G card, only half the available space is usable.
Normally growing a partition is a two step process:
(for shrinking you reverse the steps)
This turns out to work mostly the same for an encrypted partition.
❯ sudo parted /dev/mmcblk0
GNU Parted 3.2
Using /dev/mmcblk0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) pr
Warning: Not all of the space available to /dev/mmcblk0 appears to be used, you can fix the GPT to use all
of the space (an extra 63145984 blocks) or continue with the current setting?
Fix/Ignore? fix
Model: SD GC2QT (sd/mmc)
Disk /dev/mmcblk0: 64.0GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 17.8MB 16.8MB fat32 android_meta
2 17.8MB 31.7GB 31.7GB android_expand
(parted) resizepart 2 100%
(parted) pr
Number Start End Size File system Name Flags
1 1049kB 17.8MB 16.8MB fat32 android_meta
2 17.8MB 64.0GB 64.0GB android_expand
(parted) q
Information: You may need to update /etc/fstab.
First map the encrypted partition using the new size:
dmsetup create crypt6 --table "0 `blockdev --getsize /dev/mmcblk0p2` crypt aes-cbc-essiv:sha256 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 /dev/mmcblk0p2 0"
Now resize the filesystem:
❯ sudo resize2fs /dev/mapper/crypt6
resize2fs 1.45.0 (6-Mar-2019)
Please run 'e2fsck -f /dev/mapper/crypt6' first.
❯ sudo e2fsck -f /dev/mapper/crypt6
e2fsck 1.45.0 (6-Mar-2019)
Pass 1: Checking inodes, blocks, and sizes
Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/crypt6: ***** FILE SYSTEM WAS MODIFIED *****
/dev/mapper/crypt6: 47391/1933312 files (1.0% non-contiguous), 3824859/7732731 blocks
❯ sudo resize2fs /dev/mapper/crypt6
resize2fs 1.45.0 (6-Mar-2019)
Resizing the filesystem on /dev/mapper/crypt6 to 15625979 (4k) blocks.
The filesystem on /dev/mapper/crypt6 is now 15625979 (4k) blocks long.
❯ sudo dmsetup remove crypt6
❯ sync
Your new card should now show full capacity in android.
For some reason in my case Android shows a fictitious extra 64G of used space and capacity, maybe because it’s confused by the device having different metadata.
Sparklines
bee minder track things
FirefoxOS
http://www.w3.org/WAI/IndieUI/ http://www.w3.org/WAI/intro/aria
https://etherpad.mozilla.org/openscience https://en.wikibooks.org/wiki/Citizen_Science
simple
Quit! podcast http://5by5.tv/quit
Lean Startup Eric Kies
sos.oregon.gov $100
Three collaborative spaces I know well enough to recommend, written up in response to a friend’s search.
Open floorplan collectivist space, with lots of community facilities including a full kitchen and side rooms with whiteboards and phones. Some have reported eerie quiet in the main area during the day weirds them out but you should try it and see for yourself.
They host an all-day open house once monthly.
A mix of open plan and “fireboxes” which are little cubes (what a bad name right?). Less techie, more traditional professional services vibe. Their “5 day a month” membership level includes access to conference rooms. Also in a cool neighborhood, which is nice for thinking/walks.
A Portland institution and that has helped filled the void (along with CoAg) left when Cube Space closed in 2009.
I moved my Documents folder into camli (on my home ARM server) and have been accessing it via FUSE for 10 days or so.
Overall, the FUSE filesystem has been unstable.
This is probably an easy fix, I haven’t looked at the code yet.
This is painful over slow links. Not only does the writing app (VIM/sublime) block, but also things tmux (not sure why, maybe only one thread in FUSE somewhere?)
Support for excluding specific files/operations (xattrs) would require a local buffer of some sort, either a blob store or a real filesystem overlay, which could be flushed when FUSE is unmounted.
One short term bit which would help is a switch to disable xattr support at the FS level.
Also, tell apps to write their temp files elsewhere and disable atomic writes;
Sublime Text: disable atomic writes
Provide a buffer to keep files before they are sent to the server.
don’t try to store files until they’re unchanged for a little while. This should filter out scratch files like *.crdownload
camli personal storage system start server cammount import everything similarity to git recent importer rewrite (web config ui, store api creds and user auth tokens as data objects) importer rewrite foursquare
My Beaglebone (shipped March 15) came stock with Angstrom, which I replaced with the official BB Debian image. Beaglebones are now shipping with Debian preinstalled.
The Beaglebone Debian image comes with a bunch of useful stuff (Git, tmux) but it only leaves about 120MB for installing application packages, log files, etc.
To see a list of manually installed packages sorted by size, open aptitude and
list of possibly unnecessary packages:
My home router is DD-WRT. I highly recommend using it over stock vendor firmwares.
3179, 22git clone ...cd /usr/local/go/src (or wherever your Go source is)sudo GOOS=linux GOARCH=arm CGO_ENABLED=0 ./make.bash --no-cleango run make.go -arch=arm -os=linuxgo run make.go -arch=arm -os=linux && rsync -avz bin/linux_arm/camlistored $USER@$HOST:/usr/local/bin/https://wiki.thingsandstuff.org/Grid/rhythm
http://fourkitchens.com/blog/2013/10/14/singularity-grids-without-limits
I am using Harp to build my pages. However, it is sub-optimal when it comes to CSS and JS builds.
Maybe something like this makes sense:
Unfortunately, Harp doesn’t support –watch for compile mode. This needs to be fixed I think.
An ICC profiles that “fixes” CIELAB (called Uniform Perceptual Lab) http://www.brucelindbloom.com/index.html?UPLab.html
Thinking about:
including sorting the private stuff from the public
what about a UI for working with snippets “in context”
Workspaces
i am thinking about camlistore
this stream of consciousness writing feels good
t:Wisdom #mantra IGNORE EVERYTHING
I need a systematic way of thinking
a way to reconcile the stream of time with the world of abstract objects
stream
lexer
character ranges (UTF8) (objects in the stream)
creational grammars
picking objects out of the stream
stream as data type
What did I get out of brainstorming today?
Ok, let’s try yesterday’s prescription.
Feeling: like a storm has passed woke up to birthday wishes (i forgot it was my birthday)
Yesterday I rode my bike a bit, but still didn’t get enough exercise I think since I felt weird in the evening.