Saturday, June 01, 2019

Speed Scripting - EXIF, JSON, and OpenBadges

As OpenBadges have become more commonplace, I find that I haven't earned them all in a single identity; some were awarded through my work email address, others via my primary personal email address, and still others to yet another personal email address.  This led to a bit of confusion in a recent course of study, when I accidentally completed the prerequisite work under multiple email addresses.  To resolve that problem (and ensure that I don't do that in the future), I needed a tool to display the email address to which each of my badges had been awarded.

Now, there are a few complicating factors:
  1. This information is stored (in the badge PNG image) as metadata, namely a JSON array.  We don't yet have any standard/common Linux command-line tool for parsing JSON, and I like to write scripts with as few dependencies as possible.
  2. The OpenBadges specification has been around long enough to go through a few versions, and the location of the information has changed from version to version; that means I can't do a straightforward "it's always in field 27" text-based extraction.
  3. For verification purposes, the recipient's email address is NOT stored as plaintext; rather, a SHA256 hash of the address is embedded.  So, I'll need to know the SHA256 hashes for each of my email addresses used in earning badges; I used this free SHA256 Hash Generator to grab the necessary hashes. 
First, I installed exiftool (which is available for most Linux distributions - check your package manager) to extract the PNG metadata.  The OpenBadges JSON array is returned as a single string element, like so:
Openbadges                      : {"@context":"https://w3id.org/openbadges/v1","type":"Assertion","id":"https://www.youracclaim.com/api/v1/obi/badge_assertions/c1396602-e7c9-4213-a769-7d2735fb0f28","uid":"c1396602-e7c9-4213-a769-7d2735fb0f28","recipient":{"type":"email","identity":"sha256$cb6c6af3883a452da960b6cb88630b221888e7afce0ff612a483351664bb7dd0","hashed":true},"image":"https://acclaim-production-app.s3.amazonaws.com/images/84ac9eff-b8a2-4683-846b-f59887a73801/Python%2B101%2BData%2BScience.png","evidence":"https://www.youracclaim.com/badges/c1396602-e7c9-4213-a769-7d2735fb0f28","issuedOn":"2017-12-12T23:57:06.000Z","badge":"https://www.youracclaim.com/api/v1/obi/badge_classes/91e77961-2bcc-4b3a-9ff8-9333921bb2c4","verify":{"type":"hosted","url":"https://www.youracclaim.com/api/v1/obi/badge_assertions/c1396602-e7c9-4213-a769-7d2735fb0f28"}}

The only piece of this that interests us (in this case) is this:
sha256$cb6c6af3883a452da960b6cb88630b221888e7afce0ff612a483351664bb7dd0
Now, this series is entitled "Speed Scripting", so I'm not about to write a full JSON parser.  *chuckle*  Given that this is basically a string of text output, I'm going to use grep to pull the Openbadges string from exiftool's output and employ awk to iterate through the fields and print only the field containing "sha256", using the double quote as my field delimiter.  (I'm using " as my field delimiter so that I don't have to go through an extra step to strip the quotes from the output string.)  The resulting string will be stored in the $hashaddr variable:
BADGEDIR=/home/wmorgan/badges
for file in $BADGEDIR/*.png
do
  hashaddr=`exiftool $file | grep Openbadges |
  awk -F'\"' '
     {for (i=1; i<=NF; i++) {
        if ($i ~/sha256/) print $i;
     }}'`
done
(Note that I had to use a backslash to escape the doublequote character in the -F option to awk.)

That just outputs all the SHA256 hashes...but that's only the halfway point; I need to map those hashes to their corresponding email addresses.  So, I'll add my "known hashes" that I generated earlier (note that we must prepend "sha256$" to each hash string and escape the $ character):
MYHASHEDADDR1="sha256\$caceb56680cabe892cdc5b903b2cbaf9ec3462699dff9094cc25ad27ef824aaa"
MYHASHEDADDR2="sha256\$cb6c6af3873a452da960b6cb88630b221845e7afce0ff612a483351664bb9bd0"
MYHASHEDADDR3="sha256\$2a196380936706eac9c60c638be63409be9eb2728ed0302188ce2d899ed22afb"
and use the shell's case construct to handle the comparison and output:
case $hashaddr in
     $MYHASHEDADDR1)
echo wesbo@email.address.edu
        ;;
     $MYHASHEDADDR2)
        echo wessinator@second.email.org
        ;;
     $MYHASHEDADDR3)
        echo bigwes@yetanother.emailaddr.com
        ;;
     *)
        echo Unknown
        ;;
  esac
Finally, I need to print the name of the badge with its corresponding email address, so I'll extract that from the filename with basename and print it at the top of my do loop:
echo -n `basename $file .png`": "
So, here's our finished product:
#!/bin/sh
#
# dispbadgeaddr - show email address to which OpenBadge was issued
# requires: exiftool
# prerequisite: MYHASHEDxxxxx contains SHA-256 hash of email address
MYHASHEDADDR1="sha256\$caceb56680cabe892cdc5b903b2cbaf9ec3462699dff9094cc25ad27ef824aaa"
MYHASHEDADDR2="sha256\$cb6c6af3873a452da960b6cb88630b221845e7afce0ff612a483351664bb9bd0"
MYHASHEDADDR3="sha256\$2a196380936706eac9c60c638be63409be9eb2728ed0302188ce2d899ed22afb"
BADGEDIR=/home/wmorgan/badges
for file in $BADGEDIR/*.png
do
  hashaddr=`exiftool $file | grep Openbadges |
  awk -F'\"' '
     {for (i=1; i<=NF; i++) {
        if ($i ~/sha256/) print $i;
     }}'`
done
  case $hashaddr in
     $MYHASHEDADDR1)
echo wesbo@email.address.edu
        ;;
     $MYHASHEDADDR2)
        echo wessinator@second.email.org
        ;;
     $MYHASHEDADDR3)
        echo bigwes@yetanother.emailaddr.com
        ;;
     *)
        echo Unknown
        ;;
  esac
done
and here's a bit of its output:
ibm-clm-for-safe-level-1.1: wesbo@email.address.edu
ibm-cloud-essentials: wessinator@second.email.org
ibm-cloud-kubernetes-service: wessinator@second.email.org
If I wanted to polish this a bit, I'd rewrite the core exiftool/grep/awk logic as a standalone shell function, then add the capability to specify particular PNG files on the command line or use -a to iterate through all the badges.  For now, though, I can just use the script as is and use grep to extract info as needed...
$ ./dispbadgeaddr | grep blockchain
ibm-blockchain-consulting: wessinator@second.email.org
ibm-blockchain-essentials: wessinator@second.email.org
ibm-blockchain-scale-v1: wessinator@second.email.org
interskill-blockchain-foundations: wessinator@second.email.org
Total time - 10-15 minutes...and that's what speed scripting is all about.

Wednesday, August 15, 2018

Speed Scripting - Making an Animated GIF with ImageMagick

Those of you who follow me on Twitter probably know that I'm something of a geek-of-all-trades; I love reading and learning.  When my employer, IBM, entered the world of Open Badges for employee credentials, I jumped in with both feet.  We're encouraged to display our badges in email signatures, online, et cetera...but what is one to do when one has earned 50+ badges in a wide range of areas/disciplines?  Now, obviously I can't put that many badges in an email signature, and a static display can only do so much...so the thought occurred to me, "I should be able to turn these into an animated image!"  Time for some shell scripting...



Let's consider the source material:

  • Our badging provider provides badges as 352x352 PNG images, so I don't have to worry about resizing individual images; I can treat them all the same way and resize the end product as needed.
  • I'm not worried about the PNG metadata in this particular case, so converting to another format isn't a big deal.
  • I could go with an animated PNG (APNG), but (a) not all browsers display APNGs properly, and (b) animating PNGs can result in images significantly larger than comparable animated GIFs

At this point, I'm ready to start playing with ImageMagick, a wonderfully complete (and open source!) image manipulation package.  ImageMagick handles 200+ image formats, and is available as both a command-line toolbox and libraries/interfaces for a wide range of programming languages.


The actual construction of the animated GIF with ImageMagick is straightforward - it's a one-liner:

$ convert *.png -frame 5 -dispose Background -set delay 150  allmybadges.gif

The options put a frame around each image, dispose of the previous image before displaying the next one, and set the delay (in hundredths of seconds) before adding them to the end product, allmybadges.gif.


Wait, though...oh, I don't like that.  Since many badges have similar names (e.g. task-level-1.png and task-level-2.png, ibm-iot-whatever1.png, ibm-iot-whatever2.png, etc.), my use of a wildcard puts them in alphabetical order...which is kind of boring when 2-3 consecutive badges share color schemes or other design elements.  So, what I really want to do is randomize the badges before animating them.  Well, bash provides $RANDOM as an environment variable, yielding pseudorandom integers between 0 and 32767; that should be sufficient for me to play with filenames.

I wound up with this script:

#!/bin/bash
#
# badgeanim - create animated GIF of all PNG badges
#
WORKDIR=/home/badges/animate
BADGEDIR=/home/badges

cd $WORKDIR                          # move into working directory
rm $WORKDIR/*.png                    # blow away any old PNG files


for file in $BADGEDIR/*.png          # grab copies of current PNGs
do
cp $file ./$RANDOM`basename "$file"` # prepend $RANDOM to filename
done

convert *.png -frame 5 -fill snow2 -background snow2 -dispose Background -set delay 150 -set dispose Background allmybadges.gif                                         # ImageMagick builds the GIF

rm $WORKDIR/*.png                    # blow away leftover PNG files
exit


The results can be seen at the top of this article and in the "My OpenBadges" widget on the right margin of the page.  (Clicking on the animated GIF to the right takes the viewer to a static display of all badges.)  I could resize the end product for use elsewhere, like so:

$ convert allmybadges.gif -resize 200x200 allmybadges-200x200.gif

Check out ImageMagick - it's definitely worth your time if you do anything at all with images.

Saturday, August 11, 2018

First Steps in node.js - Fortune Cookies!

I've been doing a lot of learning around cloud architectures, microservices, and the like, so I thought it was time to start learning JavaScript and node.js.  I started working with IBM's SDK for node.js, but soon learned that IBM has deprecated its proprietary SDK in favor of the community SDKs. I installed the latest/greatest community SDK (v10.8.0) on my Ubuntu Linux system.

Some months ago, I wrote a Twitter bot that delivers a random fortune-cookie quote/saying every 6 hours. (It's @CollectedQuotes, if you'd like to follow it.)  Well, what better example could I adapt as a simple service in node.js?  Here we go, short and sweet (click images to enlarge)...

Here's the source code:


Here's the (very simple) browser page it generates:
Here's the 404 response to any URL request other than the root:
Finally, here's the console log:

It might be clunky, but it works!  If you'd like to play with this, you can download the JavaScript source and the fortune-cookie file.