Docker Slow Ext4 Partition

I’ve recently started using WSL and have fallen in love with it.Linux on Windows, The best of both worlds combined!One of the major drawbacks however is that file access in wsl through the Windows filesystem is slooooooooow...

/ has Docker images (via /var/lib/docker) which is the concern: docker can fill the root partition. /srv has Jenkins build results (large), zuul-merge repositories ( 25GB), integration.wikimedia.org docroot (small) We would want to shrink the volume group and create a new one for Docker images which would be at /srv/docker. To do this, right-click on the partition and select “Mount to Disk Letter.”. Select Unmount after you are done browsing the folder and accessing the files that you need. You will then be able to choose the disk letter you want to mount the partition to. Press “Mount” and you’ll be able to access the Ext4 partition from Windows Explorer. Mar 24, 2016 Additional environment details (AWS, VirtualBox, physical, etc.): Physical Environment (8GB RAM, Core i5-4590, ext4) Steps to reproduce the issue: Install openSuSE 13.2. Run Iometer benchmark on the host. Run Iometer benchmark on a container based on SuSE. A data volume was used as the partition were to execute the benchmark. Prune unused Docker objects. Estimated reading time: 5 minutes. Docker takes a conservative approach to cleaning up unused objects (often referred to as “garbage collection”), such as images, containers, volumes, and networks: these objects are generally not removed unless you explicitly ask Docker to do so. To create an ext4 file system in a partition /dev/sdb1: # mkfs.ext4 -L var-lib-docker /dev/sdb1. The partition must already exist. Use a utility such as fdisk (MBR partitions) or gdisk (GPT partitions) to create one if needed.

I happen to have a lot of content on an external hard disk formatted to ntfs.Most of the software that accesses this content was moved to Docker in wsl some months ago which is awesome because I no longer need to run this software on ‘my computer’.More importantly I don’t need to have any of the dependencies installed on my computer!

But the content was still loaded through Windows and it was starting to bother me how slow my workflow had become.I was starting to realize the benefits might not be outweighing this drawback.

But then I thought “Hey why don’t I just convert this disk to Ext4 and mount it directly to WSL?”!Well the answer is that currently you can’t load external disks in WSL.Luckily I was not the only one that thought this might be useful.Microsoft is currently working on enabling mounting disks in WSL natively and the feature is available for preview through the Windows Insiders Program.

Warning

If you decide to continue and join the Insiders Program, realize that this contains cutting edge Windows updates and can potentially break your system in an update.

You’ve joined the Dark Side Insider Program

So you’ve signed up for the windows insider program and you’re done installing Windows Updates. You have the fancy new Windows features like the new explorer icons (they look pretty good by the way). How do you now mount your disk?

Well that part is actually pretty simple!

First identify your Windows ‘physical’ diskpath by starting Powershell as administrator and using the command wmic diskdrive list brief. The output should look a little something like this.

In my case my disk is WD Elements 107C USB Device at .PHYSICALDRIVE5.

As you can see it has only one partition. I happen to know it is of type Ext4 which the WSL kernel can natively understand so all I have to do to mount my disk is run

And now the disk will be mounted in my WSL at /mnt/wsl/PHYSICALDRIVE5

Going deeper

If you have more than one partition you will have to specify the partition number you want to mount like so

This does mean you need to know which partition contains your Linux filesystem.

To find that out we can mount the disk as ‘bare’ which basically means that Windows passes through the disk to WSL but does not actually try to mount it to the Linux filesystem.

Then identify the device number for your hard disk with lsblk

The output will look something like this:

Partition

If you don’t immediately recognize your disk from this list based on the size you can use the following command to get all physical drives attached:

sudo lshw -class disk

The output should look like this:

Here you can find the disk by it’s name and model which you can compare with the wmic diskdrive list brief command from Windows. See the disk is listed as /dev/sdd for me.

So now we know which device is our disk but we still don’t know anything about the partitions on the disk. For that we need to go back to lsblk. Find our disk in the list based on the device id (/dev/sdd). Now for every partition listed for this device you have to run blkid <device-id><partition>. In my case since I have only one partition this is as simple as sudo blkid /dev/sdd1

If you have multiple partitions you will have to run this command for each of them untill you find the partition you need.

As you can see in the output my partition is of type ext4 which means it is supported.

At this point you can go back to mounting the partition in Windows and follow the first step.

Bonus points

Docker

As we saw earlier WSL mounts the partitions automatically in /mnt/wsl based on the physical device name and the partition id. I wasn’t particularly happy with this default as a physical device name tells me nothing about which disk it is. No matter though because as we learned we can mount the disk bare and have the device show up in Linux anyway. Using this information and the information from blkid /dev/sdd1 I realized that I could simply mount the partitions in Linux instead!

First I created a mount point (directory) in /mnt called data with mkdir /mnt/data.

Then I edited /etc/fstab (the ‘mount configuration’ file in linux) with nano: sudo nano /etc/fstab and added a line to mount my Linux partition on /mnt/data.

The entry looks like this:

UUID=6d7e2d05-454d-d701-605c-2d05454dd701 /mnt/data ext4 defaults

Notice that the UUID comes from the output of sudo blkdid /dev/sdd1, the next part of the line is where to mount. Then come the filesystems and the mount options which I left as ‘defaults’. Now run sudo mount -a and you’ll have access to your drive from WSL!

And there you have it, a completely dedicated disk for WSL.

Docker Slow Ext4 Partition Download

I have not yet figured out how to mount this automatically on startup due to needing to run the command wsl --mount .PHYSICALDRIVE5 --bare as administrator (could probably use a scheduled task for this) but that is a small price to pay for the awesome performance and increased WSL disk space I now have. Enjoy!