Containerize as-is: --preserve

vrtmv migrate --format container builds an image by installing the target distribution's packages. Software the target has no package for is not in that image. --preserve <path> carries such software into the image unchanged, with the shared libraries it loads.

The two are different claims. Translated software can be verified equivalent. Preserved software is attested as the source's bytes and nothing more: verify grades the image preserved-as-is, never equivalent, and does not say the program runs.

Available from 2.2.0. Container output only: there is no --preserve for the Ansible, Terraform or image formats.

When to use it

Use it for software the migration cannot install on the target:

• an install no package owns, under /opt, /usr/local, /srv, /home or /root: a vendor binary, an in-house application;
• a file of an installed package the target has no package for: an interpreter the target distribution dropped, a vendor package the Index has no row for.

Do not use it for software the target carries. vrtmv refuses that, and names the target package. Installing the target's package is the path that can be verified equivalent and that the target's package manager will patch.

Prefer the target's own package where one exists under another name or in another repository. For an install no package owns, vrtmv does not look for one: it establishes that the software is unpackaged on the source, which is a narrower fact than the target having nothing for it.

It is not the path for data. A directory of user files is vrtmv data's, and a database's data directory is refused here and goes through the database export.

How vrtmv tells you a source has such software

The untranslatable-software feasibility check names it, from 2.3.0. vrtmv assess prints it under Container feasibility:, and vrtmv migrate --format container prints it on stderr and records it on the attestation. See the check for what it reads.

It is a decision for you, and it prints the --preserve argument for each thing the flag would accept. vrtmv never sets the flag. A run without --preserve writes the same translated Containerfile as before and says what is not in it.

What it tells you is narrower on assess than on migrate. assess calls no API, so it names software no package owns and judges no installed package. migrate has the plan and also names packages.

What you run

vrtmv migrate --root <src> --target rocky9 --format container \
    --entrypoint "/opt/vrtmv/preserved/bin/opt/acme/bin/acmed -f" \
    --preserve /opt/acme \
    --preserve /usr/bin/python2.7 \
    -o out
vrtmv data preserved --attestation out/vrtmv-attestation.json -o out

The flag is repeatable. Each path resolves to one unit: the whole unowned install the path is in, or the whole package that owns the file. Without --format container the flag is an error.

What the run produces:

• The Containerfile is the translated image as before, with a fenced block added, opening with # ---- vrtmv PRESERVED AS-IS: not translated, not attested equivalent ----. Everything that translates is still installed from the target's packages.
• A preserved file under one of the five directories above keeps its path. Every other preserved file, and every library and the loader it needs, goes under /opt/vrtmv/preserved/sysroot, so the source's libc never replaces the target's.
• A generated script, check-kept-paths.sh, and two lines in the block that copy it to /opt/vrtmv/preserved/ and run it before any preserved file is copied to a path it keeps. The build stops when the image already has something at one of those paths.
• One generated launcher per preserved executable, under /opt/vrtmv/preserved/bin, runs it under the carried loader with the carried libraries. Your --entrypoint is not rewritten; point it at the launcher.
• Two image labels, com.vrtmv.preserved-as-is and com.vrtmv.preserved-units, and a marker file at /etc/vrtmv/preserved-as-is.
• The attestation records each preserved file's path, size, mode and sha256.

migrate copies no file contents. vrtmv data preserved writes a staging script and a sha256 manifest, and copies nothing either. You run the script against the source root. It copies each attested file into the build context, hashes the copy, and stops at the first one that does not match the attestation.

A script tree needs its interpreter in the image as well: the target's package for it, or the interpreter named to --preserve as its own unit.

What --preserve refuses

Each refusal names its reason, exits non-zero and writes no Containerfile.

• A path owned by a package the target carries, whether through a curated translation, a same-name package in the target's repositories, or a repository the default set leaves out (the message names --target-repos).
• A package that is part of the source distribution's own base.
• A package whose translation installs nothing for a reason the plan does not show to be the target having none.
• A path in no named install and owned by no package: a file directly under /opt, a path under a category directory of /usr/local such as /usr/local/bin, a path under a dot directory, or a path outside the five directories.
• A unit that is inside a database data directory, or contains one.
• A path that is a symbolic link, or goes through one. Name the path it leads to.
• A source whose package database could not be read, or was read in part. "No package owns it" cannot be told from "could not ask".
• An SSH collection, for a unit whose files were not fetched. An SSH collection lists /opt, /usr/local and /srv and does not fetch them, and does not collect /home or /root. Use --root or --image.
• A source the feasibility checks block or could not determine. --preserve is not a way around a blocker.
• A program that is run by a loader other than glibc's, that needs a library the source does not have, or whose library list depends on an input that was not evaluated.

Character-set conversion: what migrate tells you

From 2.3.0. Whenever a launcher sets GCONV_PATH, migrate --preserve reads the preserved files for a use of glibc's iconv and says which of three things it found. It says it in three places, so you see it before you build or deploy: on stderr, in a comment in the Containerfile block, and in the attestation under preserved.closure.gconv.iconv_use.

What to do with each:

• With a use, the preserved program converts character sets, and in the image it does so with the old glibc's modules, as on the source. Find what it starts. A program from the target image that it starts and that converts is the case that was measured to hang or crash. Start those with GCONV_PATH unset. A preserved shell script that runs iconv runs the target image's iconv unless you preserved one, and that is the measured hang.
• With none found, the same limit still applies to anything the preserved program starts. The launcher sets the variable whatever the program does.
• With could-not-tell, treat it as a use until you know otherwise.

What is read: the dynamic symbol table of each preserved file and of each carried library that is not the old glibc's own, and the text of each preserved file that starts with #!, up to 1 MiB.

What is not read, so what no use of iconv was found does not say:

• which programs a preserved program starts, and whether those convert;
• a conversion glibc makes without iconv being called. Its multibyte and wide-character functions use the same modules in a locale whose character set needs one, and so does fopen with ccs=;
• anything a program loads at run time from outside the preserved files;
• what a script's interpreter converts. A script without the word iconv is always could-not-tell;
• a statically linked program, which is always could-not-tell;
• a text file that does not start with #!, such as a shell fragment another script sources. It is not counted either way, with the word iconv in it or without;
• a program that converts with GNU libiconv or ICU. Those do not read GCONV_PATH and are not reported.

A script is matched on text, so a comment that mentions iconv counts as a use.

The claim the output carries

verify never returns equivalent for an image with preserved files. When everything translated matches and every preserved file is in the image byte-identical, the verdict is preserved-as-is and the exit code is 5. It is not a pass. Exit 0 still means equivalent and nothing else, and there is no flag that maps 5 to 0. A pipeline that has decided to accept a preserved image gates on 5. With preserved files, verify never exits 0 or 3: a target-limited report exits 1, and under --require the preserved files are always a required class.

What is attested about a preserved file is that the file in the image has the same sha256 as the file on the source. Not that it runs, and not that it behaves as it did.

What it does not do

None of the following is softened by the image building.

• It does not say the program runs on the target.
• The library list is a floor. Libraries a program loads at run time with dlopen are not in it, and the attestation records run-time loading as unmeasured.
• Name and user lookups can load the target's libraries into the preserved program. The image carries the old glibc's own name-service modules, so lookups answered from files and DNS work as on the source. Which modules are tried comes from the target's /etc/nsswitch.conf. When a lookup falls through to a module only the target has, the target's copy is loaded into a process running on the source's old libc. vrtmv does not test that and does not say it is safe.
• GCONV_PATH is inherited by child processes. The launcher sets it so the preserved program uses the old glibc's character-set conversion modules, and every program the preserved program starts inherits it. A target program started that way is given the old modules while running on the target's glibc. Measured with a CentOS 7 source on Rocky Linux 9: the target's iconv converting ISO-8859-15 to UTF-8 did not return and was killed at 20 seconds. It is not one character set. Of the 89 names that image's iconv lists, 41 had a conversion from the character set still running at 3 seconds and 16 had one that crashed, UTF-16 and UTF-32 among them. Conversions to a character set were not affected. The same conversion also hung on a Debian 12 image. It reaches every target program below the preserved one, however many steps down. A target program that starts a preserved one is not affected. This is a limit of carrying the old glibc and it is not going to be removed: the ways found of removing it change a carried file or rest on loader behaviour glibc does not document. Start target programs from a preserved program with GCONV_PATH unset. migrate tells you whether the preserved files convert at all; see Character-set conversion. The launcher does not limit the variable to the program it wraps. With target programs started that way the same conversion returned the right bytes. Set-user-ID and set-group-ID programs ignore the variable, which is glibc's rule. That was not measured.
• Ownership and set-user-ID bits are lost at COPY, and directory modes at staging. File modes are carried; owners, set-user-ID and set-group-ID bits and directory modes are not. A file owned by a service account on the source is root-owned in the image. A private directory becomes 0755, because the staging script creates directories and does not copy them. migrate prints a line for each preserved file that has such a bit. File capabilities and other extended attributes are not recorded.
• The launcher covers the program it wraps. A child process started by absolute path runs under the target's loader. A set-user-ID bit has no effect on a program run through an explicit loader. Absolute paths inside a binary are not rewritten.
• The architecture is checked at build time, not at migrate time. A RUN line in the block compares uname -m inside the build with the architecture of the preserved files and fails the build when they differ. A Containerfile for the wrong platform is still written. The comparison is the kernel's machine name, so a 32-bit x86 image built on an x86_64 kernel passes the check and is mislabelled. The same may hold for 32-bit arm on an aarch64 kernel; that was not measured. Without emulation, a build for another platform stops at that line with the runtime's own error and not vrtmv's message.
• A preserved file under /opt, /usr/local, /srv, /home or /root keeps its path, and COPY would replace a target file at the same path. The build checks for that and stops: the log has a line vrtmv: <path> is already in this image, and the preserved file from the source would replace it for each such path, and a build that runs the check and passes has a line vrtmv: none of the <n> path(s) the preserved files keep is in this image. A rebuild that takes that step from the layer cache does not run it again and prints nothing from it: podman's log has --> Using cache under the step. So the line missing from a rebuild's log does not mean the check was left out. Build with --no-cache to see it. A cached step belongs to the image it ran on: when the base tag was pointed at an image that has the file, the rebuild ran the check again and failed. It also stops when a directory above a kept path is in the image as something other than a directory. It does not check a merge: when the image has the same directory with other files in it, the build passes and the preserved files land beside them. It does not look at /opt/vrtmv/preserved or the marker, at anything added to the image after the block, or at a build that runs as a user other than root. The check runs in the image, so a build for another platform without emulation stops at that line with the runtime's own error. The script uses only shell builtins and was built on all 19 base images in Container base images with podman, and on one with kaniko. Docker was not run. The script stays in the image at /opt/vrtmv/preserved/check-kept-paths.sh.
• The host kernel is not the source kernel. An old userland can depend on kernel behaviour a current host has turned off. vrtmv does not test for it.
• Preserved software is unpatched. The source's old libc and other libraries are in the image outside the target's package manager, and dnf update will never touch them.
• verify --ssh cannot grade a preserved image, because an SSH collection does not fetch the preserved bytes. The result is inconclusive. Use --root or --image.
• Kernel modules, one process per image and the feasibility checks are unchanged by this flag. An operator-declared module still refuses the output with --preserve.
• No licence check. vrtmv records what was carried and asserts no right to copy or redistribute it.

Older verify builds

• Released verify builds before 2.2.0 cannot read a preserve attestation. It is written so that 2.1.0 and earlier stop with an error. Verifying a preserved image needs 2.2.0 or later.
• An older verify given a plain attestation for the same source still reads a preserved image as it would any other. A plain attestation is one from a run without --preserve, or a preserve attestation unwrapped by hand. 2.1.0 then prints equivalent and exits 0, and the signing service signs it, because it signs a digest and does not see the verdict. Nothing in a later release can change a binary that is already released. verify from 2.2.0 returns inconclusive in that case, from the marker file.
• The boundary is keyed on the attestation and on the marker. An attestation with the preserved key removed is graded as a plain migration, and an image with the marker deleted is not recognised as preserved. Both are tampering.