Shell Overlay Icons – The Space Wars

For some of us there has been a quiet war raging inside the Windows registry. The fight is over your shell overlay icons and their priority. I fought back!

The Problem – Shell Overlay Icons Limit

Amazingly, even going into the modern era of Windows 10 in 2017 this is still an issue.  The issue is that many tools want to make use of shell overlay icons but Windows only has 15 “slots”.  It’s safe to assume that many of these tools would use this Windows feature for an “at a glance” method of displaying state.  This also means that a single tool wont just use one icon but many to display various states of files and folders.  For example the little screen shot below shows two shell overlay icons, one is green denoting a committed state and one red highlighting an uncommitted state.

an example showing shell overlay icons

As our ever-more digitally connected world evolves, more and more tools want to make use of this feature.  I’m sure most of you reading this have at least a couple of tools that do this.  However, the list for me goes on and on.  Tools like OneDrive, Dropbox, Git and SVN and … you get the picture.

Given that Windows only uses the top 15 entries in the registry, I have over 30 listed.  Dropbox alone brings 10 to the table, so what are we supposed to do?  As you can see 15 isn’t going to go particularly far given this scenario.  Whilst these shell overlay icons are useful in some scenarios they probably aren’t the best solution to the problem anyway.  There is a good discussion here on the problem from Raymond Chen who works on the Shell Team at Microsoft.

The Space Wars

What the hell am I on about anyway – Space Wars?  Many of these tool vendors, Dropbox I’m looking at you in particular, have started a kind of war inside your registry.  Aside from any chuckling ‘nix users chortling about the fact Windows even has something as insane as a registry we still have to deal with it.  Actually, to be fair a lot of the issues with the registry is down to my fellow developers abusing it but that’s a whole other story for another time.

So, what exactly is the problem?  When you take a registry entry key name like “DropboxExt01”  you’d expect that come before “DropboxExt02”, right?  Well, kinda.  If I rename “DropboxExt02″ to ” DropboxExt02″ (notice the leading space) ” DropboxExt02″ now comes before “DropboxExt01”.  And thus was born the space wars …

Each vendor thinks their tool is the most important, obviously.  So they’ve taken it on themselves to start appending their shell overlay icons key names with ALL THE SPACES, ARRRRR!!  Forcing their entries to top of the tiny selection that Windows will actually bother to use.

I Fought Back!

I’ve lost count of the number of times I’ve fired up regedit in order to fix this insane situation.  Countering a vendors update install that “fixes” that tools entries (read: appends more spaces).  “Why wouldn’t you want our icons to work”.  Well dear Dropbox your tool isn’t the center of my world, in fact I’ve nearly uninstalled you as this is such an annoyance.

Anyway, I’d just had enough yesterday.  So I wrote a tool.  Say hello to Overlay Ninja …

shell overlay icons - overlay ninja

Okay, okay it doesn’t look that great (yet) but I knocked it up in a few hours.  Now I can fix this problem easily in a couple of clicks and without going anywhere near regedit.  The source code for this is all up on my GitHub page.  It’s under GPLv3 so if you make any improvements please do submit a pull request so we can all benefit.

You can set priorities by application or by each individual shell overlay icons entry which adds a lot of flexibility.  I’ve tested this as far as I can and all is working as expected.  As ever, when doing anything in the registry make a backup first.  If you’re reading this and don’t even know what the registry is, what the hell are you doing reading this? πŸ™‚  Go have a read of this before doing anything with the tool.

x86 or x64?

Inside the GitHub repository I’ve also uploaded pre-compiled versions of the tool so users without the required build tools can still use it.  Due to some architecture redirection foibles within the Windows registry you will need to use either the x86 or the x64 version of the tool for it to actually work as expected.  If your OS is 32-bit, use the x86 version, if it’s 64-bit use the x64 version (here’s how to check that).

Happy, er … Ninjaing, Ninjaning?  Ninjining?

Share

.NET Geographic Searches

I recently had to implement a localised search mechanism on an API.  Maths and Trigonometry aren’t my strengths it has to be said.  So I started looking online for pointers and background info.

Initially it might seem like you have to do some complicated maths to work out things like the size of the circle you’re dealing with etc.  But really the solution is very simple.  You know the starting points latitude and longitude and you also have a radius in a known unit, such as miles, meters or kilometers.  You’ll also have a dataset of potential target locations complete with their latitude and longitude points.

The only measurement you need is the distance from your starting point to a potential target location.  If that distance is less than or equal to the target location it’s within the circle designated by the radius.

The most accurate and popular algorithm for working this out is the Haversine algorithum.  The .NET framework has this built in.  The GeoCoordinate class inside the System.Device.Location namespace.

To use this:

var startingPoint = new GeoCoordinate(51.212213,-2.122312);
var targetPoint = new GeoCoordinate(52.212213,-1.122312);
var distanceInMeters = startingPoint.GetDistanceTo(targetPoint);

You can then convert the number of meters into whatever unit you would like to work with. Pretty simple really. The Haversine algorithm accounts for the curvature of the planet as well and has a very small margin of error.

You can find lots of implementations of this online, such as this one:

public static class Haversine {
  public static double calculate(double lat1, double lon1, double lat2, double lon2) {
    var R = 6372.8; // In kilometers
    var dLat = toRadians(lat2 - lat1);
    var dLon = toRadians(lon2 - lon1);
    lat1 = toRadians(lat1);
    lat2 = toRadians(lat2);
 
    var a = Math.Sin(dLat / 2) * Math.Sin(dLat / 2) + Math.Sin(dLon / 2) * Math.Sin(dLon / 2) * Math.Cos(lat1) * Math.Cos(lat2);
    var c = 2 * Math.Asin(Math.Sqrt(a));
    return R * 2 * Math.Asin(Math.Sqrt(a));
  }
 
  public static double toRadians(double angle) {
    return Math.PI * angle / 180.0;
  }
}
 
void Main() {
  Console.WriteLine(String.Format("The distance between coordinates {0},{1} and {2},{3} is: {4}", 36.12, -86.67, 33.94, -118.40, Haversine.calculate(36.12, -86.67, 33.94, -118.40)));
}
 
// Returns: The distance between coordinates 36.12,-86.67 and 33.94,-118.4 is: 2887.25995060711

I would definitely recommend using the .NET version though as this will be thoroughly battle tested.

Share

Docker Gotcha

I’ve just had my first Docker gotcha moment.  I was happily working away using the setup detailed in my first Docker article.  All was going well with some development work migrating a site to a new WordPress implementation.  Boom!  I had the first Windows 10 BSOD I’ve ever experienced.

The machine restarted fine without any problems.  I saw a notification that I needed to reset the password on my Microsoft account, which I dutifully did.  I then restarted Docker to carry on development and when I visited my local URL to carry on the site was new.  The development work I’d completed so far was all “gone”.  Visiting the WordPress directory I could see everything in there as it should be.  Since the Database had apparently gone I decided to rip it all down again and start afresh.  I hadn’t done much work and what I had done was all still in the file system anyway.

So I clear everything down and execute the docker-compose up -d command again.  Everything seems to go fine but the database and ui directories are still empty.  After some poking around and Googling it suddenly hit me.

When you share drives with Docker it needs your Windows credentials to mount the shared drives.  I’d just changed those which means Docker no longer had my new credentials.

An error message or warning would have been helpful Docker.

Share

Password Sensibilities

I’ve lost count of the number of organisations I’ve worked for that have adhered to the NIST (National Institute of Standards and Technology) password advice from back in 2003.  I’ve scoffed every time I’ve looked around their offices to see (a sea?) password post-it notes with various passwords written down.  Honestly, every 90 days?

Clearly they have people in charge of making security decisions that simply are not qualified to.

Finally, the author of those guidelines, NIST manager Bill Burr, has admitted that he was completely wrong.

Share

PlasticSCM – Just Don’t

If you’re considering this source control tool.  Stop.  If you’re considering this for your development team, have mercy on them and stop.

I haven’t been confused by, annoyed by, dumbfounded by or angry at a source control tool this much since I stopped using Visual Source Safe 10 years ago.  This is one of the absolute worst development tools I’ve ever had forced on me.

The hilarious thing is – it isn’t even free, people are paying real money for this!.  Believe me, if it was my choice in tooling they’d have to pay me to use it.  If you are charging for source control these days you have to be pretty damn special and plasticSCM really is not special at anything at all.

I’m not even going to go into the details of why you shouldn’t use it as I simply cannot be bothered to have this tool waste any more of my time, but I feel the need to warn others.  This tool is incoherent.  Just use Git.  There is no benefit in this tool over Git, plus Git is free.

USE GIT.

Git, Git, Git, Git, Git, Git.

Share

Docker + Windows + WordPress + MySQL + PHPMyAdmin = Nirvana

The last time I did any WordPress development was over a year ago so I no longer have PHP and MySQL installed.  I started marching off down my well beaten path.  Download PHP and MySQL, install on my machine, do dev work.  Just after clicking on the MySQL download I suddenly thought, what the hell am I doing?  DOCKER!!!!  I already have Hyper-V installed so all I need is a container and some YUMMY YAML.

I’ve used Docker once before, for around 5 minutes, so I’m a complete noob really.  So going through this process to work it all out for myself was initially extremely painful.  I failed on my first session and ended up having to walk away from it all as I just wasn’t getting it.

I did a lot of reading and Googling and just wasn’t finding the explanation I needed to grok it all properly.  There are lots of explanations of how to get this working but they all seemed to stop at the crucial point for me.  They covered off some yml to get MySQL and WordPress containers up and running but stopped there.  What about persistence?  Or deployment?  Where the hell are all the WordPress files?

Some of them seemed to demo the solution I wanted but seemed to miss out on how they achieved it, or were doing it on different platforms.  I needed a noobs Docker for Dummies walk through.  So I’m going to document what I’ve found out in the hope that it crosses some of the Ts and dots the Is for others getting started.

Docker is The New VM

Don’t get me wrong virtual machines are great and very necessary but they’re also a bit overkill a lot of the time.  This is where Docker comes in.  It still requires virtualisation technology under the hood but it’s now transparent and not directly controlled.

Microsofts own virtualisation technology is Hyper-V.  Docker on Windows uses this by default but it could be used with VirtualBox from Oracle as well.  I’ve had lots of success running virtualised OSes on top of Hyper-V and more or less utter failure using it for Microsofts own emulators, the irony here isn’t lost on me by the way.

Docker is a container technology that wraps up specific services, such as databases, runtime or any other dependencies tasks require.  It lets you run just what you need for a given task without installing these services on the host OS. Fantastic.  Lets dig in.

Installing Docker

Dead easy.  Make sure Hyper-V is installed on your Windows box (Home users you’re out of luck here btw).  Go here, download for your OS and architecture, install.  Done.

The Docker installation is a very painless process.

Check Installation

Once installed., open a command line (Windows + X -> Command Prompt (Admin)) and execute:

docker version

You should then see some version information:

Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:30:30 2017
 OS/Arch:      windows/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:51:55 2017
 OS/Arch:      linux/amd64
 Experimental: true

If you see an error mentioning the daemon, Docker may well still be setting itself up in the background.  You can also create a Docker ID on the Docker site and configure your install to use this ID though I’ve not needed to so far so cannot comments on this aspect.

Next run this:

docker info

This gives you some useful info about your Docker environment, like so:

C:\WINDOWS\system32>docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 17.06.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.36-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.837GiB
Name: moby
ID: 3RBI:664X:UXGI:FB6Y:3K7K:LEMA:BWRR:6SLX:5M7J:P66D:T4XN:L7XH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 15
 Goroutines: 25
 System Time: 2017-08-06T15:06:13.8983022Z
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Settings

It’s probably worth having a look over your Docker settings before doing anything else.  You can right click on the Task Tray Docker icon to see various options, Click on Settings first and have mooch about to see if things are setup as you’d like.

The General tab is pretty self-explanatory and controls how Docker starts up and some security settings.  You can probably leave most of these alone.

The Shared Drives page is also pretty self explanatory.  Using this page you can control which host machine drives are available to services running within containers.  Since I’m using my H drive for my projects I’ve shared this drive with Docker.  This allows processes running inside your Docker containers to access stuff on the host machine drives.

docker shared drives settings

The Advanced tab is worth reviewing.  If you have used Hyper-V before and have customised it’s setup you’ll find that Docker automatically picks up some of these settings.  I’d configured Hyper-V to put VM images on one of my large external drives so Docker will install it’s Moby Linux VM here.

docker advanced settings

I’ve also upped the available RAM Docker can use to 4Gb, my dev box has 24Gb so I’ve plenty to throw at Docker.  Since it’s running on Linux 4Gb RAM should be more than enough to keep things running at a decent speed.

The Network, Proxies and Daemon pages are fine in their default state for now.

My Requirements

I wanted a WordPress development environment with persisted data.  If Docker is restarted or the host machine reboots, I want the state of both stored.  Both in terms of the WordPress state such as themes, plugins and so on and the database.  I’m fast with WordPress development but not that fast!

So Docker should accept all the responsibility of hosting Apache, PHP, MySQL and phpMyAdmin.  Docker also takes care of all the networking and configuration of those services.  The host Windows machine exposes a drive and hosts the WordPress files and MySQL databases on it’s “normal” file system.  These files are stored in directories that are mapped into the containers which allows for simple deployment once the development is complete.

I’m sure in time and as I learn more about Docker I’ll find a lot of this can be handled better.  For now this is where I am in the learning curve and it’s working.

Yummy YAML & Docker Compose

The idea here is to produce a file that tells Docker what you want to do.  It specifies one or more images to use to create one or more containers that provide the services you need.  Images are templates for containers, much in the same way classes can be thought of as templates for objects in an OO sense.  There are some issues here in terms of terminology which I don’t get.  Although we are creating containers, in the .yml files they are called services.  It’s probably my limited knowledge here but it would be clearer to new users if they just stuck to using the same terms.

What I was struggling to understand and configure was volumes.  I’m still a little in the dark to be honest and I’m not entirely sure I have this configured in the best way.  But what I’m showing here is working and it suits the requirements I mentioned above.

Directory Structure

As I showed above I’ve shared the host machines H drive.  Within this drive I have the following directory structure:

H:\js2017
H:\js2017\database
H:\js2017\ui

In the root directory (H:\js2017) I have created a Docker compose YML file called docker-compose.yml.  This is where the magic happens.  The file contains this YAML:

version: '2'
services:
  wordpress:
    depends_on:
      - db
    image: wordpress
    restart: always
    volumes:
      - ./ui:/var/www/html
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_PASSWORD: p4ssw0rd!
    ports:
      - 8082:80
    networks:
      - back
  db:
    image: mysql
    restart: always
    volumes:
      - ./database:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: p4ssw0rd!
    networks:
      - back
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - 8083:80
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: p4ssw0rd!
    networks:
      - back
networks:
  back:

So in this file we have 3 services defined.  We have wordpress, db, phpmyadmin.  We also have a network aliased back (backend).  You could almost certainly take this file, alter the ports if need be and the volume entries and have this up and running pretty quickly.

You can see the references to the database and ui directories in the volumes declaration for the wordpress and db services.  The “.” notation is the relative path from the docker-compose.yml file in the project root directory.  These are mapped by docker to the containers internal file system running on top of Linux within the Docker virtual machine.  So anything written to the /var/www/html directory within the container ends up in the ui directory on the host machine and the same for the /var/lib/mysql directory for the databases.

The port mappings for the services are mapping host machine ports to container ports (hostmachineport:containerport).  So to view the WordPress site, I navigate to localhost:8082 in my host machines browser and this is forwarded to port 80 in the container and serves the page.

Deployment

Going over the deployment of the MySQL database to a live server is beyond the scope of this article but to deploy the WordPress site it’s just a case of taking the contents of the ui directory and uploading it to the public html directory of your web server.

I’m sure there is a better way of managing this but for now with my limited understanding of the finer details of Docker and it’s container model this works for me.  Hopefully this has gotten you started and I’m sure I’ll revisit this again in the not too distant future with some better solutions and tips.

Happy Docking!!

Share