File I/O with Containers

Overview

Teaching: 5 min
Exercises: 5 min
Questions
  • How do containers interact with my local file system?

Objectives
  • Learn how to copy files to and from a container

  • Understand when and how to mount a file/volume inside a container


Recording of the HATS@LPC2020 session (link). Note, you must have a CERN login to watch this video.

Copying

Copying files between the local host and Docker containers is possible. On your local host find a file that you want to transfer to the container and then

touch io_example.txt
# If on Mac need to do: chmod a+w io_example.txt
echo "This was written on local host" > io_example.txt
docker cp io_example.txt <NAME>:<remote path>

Note: Remember to do docker ps if you don’t know the name of your container.

From the container check and modify the file in some way

pwd
ls
cat io_example.txt
echo "This was written inside Docker" >> io_example.txt
<remote path>
io_example.txt
This was written on local host

and then on the local host copy the file out of the container

docker cp <NAME>:<remote path>/io_example.txt .

and verify if you want that the file has been modified as you wanted

cat io_example.txt
This was written on local host
This was written inside Docker

Volume mounting

What is more common and arguably more useful is to mount volumes to containers with the -v flag. This allows for direct access to the host file system inside of the container and for container processes to write directly to the host file system.

docker run -v <path on host>:<path in container> <image>

For example, to mount your current working directory on your local machine to the data directory in the example container

docker run --rm -it -v $PWD:/home/`whoami`/data sl:7

From inside the container you can ls to see the contents of your directory on your local machine

ls

and yet you are still inside the container

pwd
/home/<username>/data

You can also see that any files created in this path in the container persist upon exit

touch created_inside.txt
exit
ls *.txt
created_inside.txt

This I/O allows for Docker images to be used for specific tasks that may be difficult to do with the tools or software installed on the local host machine. For example, debugging problems with software that arise on cross-platform software, or even just having a specific version of software perform a task (e.g., using Python 2 when you don’t want it on your machine, or using a specific release of TeX Live when you aren’t ready to update your system release).

Mounts in Cygwin

Special care needs to be taken when using Cygwin and trying to mount directories. Assuming you have Cygwin installed at C:\cygwin and you want to mount your current working directory:

echo $PWD
/home/<username>/<path_to_cwd>

You will then need to mount that folder using -v /c/cygwin/home/<username>/<path_to_cwd>:/home/docker/data

--volume (-v) versus --mount

The Docker documentation has a full and very interesting discussion about bind/volume mounting using these two options. However, much of it boils down to --mount being a more explicit and customizable command. The -v syntax combines many of the options found in --mount into a single field.

“Tip: New users should use the –mount syntax. Experienced users may be more familiar with the -v or –volume syntax, but are encouraged to use –mount, because research has shown it to be easier to use.”

Key difference: If a file/directory doesn’t exist on the host:

  • -v or --volume will create the endpoint for you as a directory.
  • --mount will generate an error.

Key Points

  • Copy files to an from a container using docker cp

  • Mount a folder/file inside a container using -v <host path>:<container path>