I ran into an issue recently with my Tekton Pipelines with PersistentVolume Permissions.

I needed my pods to create files with the group write bit set. Essentially, I needed a umask of 0002, but my pods were running with umask of 0022.

I was afraid I was going to have start twiddling around with umasks for my Tekton tasks, but instead found that I could solve my issue using Linux Access Control Lists (ACLs).

ACLs are not often used on Linux, but can be quite powerful. For this article, the relevant feature is the ability to set default permissions for created files. These ignore the current processes umask and act as a directory specific umask.

The Setup

My PersistentVolumes were NFSv4 shares exported from a RHEL8 host, so I would need to set these ACLs on directories exported from that server.

Here is the normal behavior:

nfs-server#  cd /path/to/nfs_share

nfs-server# umask
0022      <-  Files should be created rwxr-x-r-x 

nfs-server# getfacl .  
# file: .
# owner: root
# group: root
# flags: -s-
user::rwx
group::rwx
other::r-x


nfs-server# touch without_default_perms_acl
# ls -l
total 0
-rw-r--r--.  1 root root   0 Jan 19 11:44 without_default_perms_acl

Now, let me set default permissions via an ACL:

nfs-server# cd /path/to/nfs_share

# Let's set our default ACLs for group and other.

nfs-server# setfacl -m d:g::rwX . 
nfs-server# setfacl -m d:o::r-X .

nfs-server# getfacl .
# file: .
# owner: root
# group: root
# flags: -s-
user::rwx
group::rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r-x

nfs-server# umask
0022      <-  Our umask remains unchanged

nfs-server# touch with_default_perms_acl
nfs-server# ls -l 
total 0
-rw-rw-r--. 1 root root 0 Jan 19 11:50 with_default_perms_acl
-rw-r--r--. 1 root root 0 Jan 19 11:44 without_default_perms_acl

As you can see, the file created AFTER setting the default ACLs ignores the umask and instead uses the default set in the ACL! New subdirectories will also inherit these ACLs.

When this directory is exported by NFSv4, this ACL is honored by the client.

For more information, see acl(5) and setfacl(1).

A word of warning, not all NFS servers will support this, and I’m fairly certain you must be using NFSv4 for this to work. I’ve tested it with a RHEL8 nfs-server, but YMMV for other nfs servers.

WHY would I need to do this?

OpenShift, unique among K8s distributions, semi-randomizes the UID and GID of it’s pods. This is a security measure that adds barriers for attackers.

When two pods need to share a PV, we need to carefully construct permissions so that they share a common group and that group has write permissions on all files.

Normally this is done by:

  1. Changing the group of the root of the NFS share to 0 (root) or a defined common group

  2. Setting the setgid-bit, so that all new files and directories inherit that group.

  3. Setting the umask to 0002, so that group read-write is set on all files.

Step #1 and #2 above are fairly straight forward, but setting the umask on every pod in every scenario can be difficult, especially if you’re dealing with pre-created container images and many images (like in a tekton pipeline)

Using default ACLs, instead of relying on umask, allows me to make one change, on the server side, that is automatically followed by all pods.