data

Migrate the payload — the application data that no package owns and translation therefore cannot carry: uploads, document roots, /opt application trees, database files. Package translation reproduces the operating system; data handles what the workload exists to serve. It runs entirely locally, calls no API, does not bill, and never moves a byte itself — it emits runbooks the operator runs.

Two phases:

# 1. discover — what non-distro data is on the source, and how to move it
vrtmv data discover --image centos7.vmdk
vrtmv data discover --root /mnt/vm --json

# 2. plan — emit runbooks for the paths you choose
vrtmv data plan --root /mnt/vm \
  --copy /srv/www --copy /var/lib/mysql -o vrtmv-data-out

discover

Walks the source's data roots (/var, /opt, /srv, /home, /usr/local, …; never /etc — that's drift's job), asks the packaging backend "does any package own this?" for every file, and groups everything unowned into candidate locations. Volatile and cache trees (/var/cache, /var/log, /tmp, the package DBs themselves) are pruned; symlinks are never followed. Spool directories under /var/spool are listed, apart from the crontab directories in /var/spool/cron and the at queue in /var/spool/at, which the scheduled-jobs check reports.

Each location is classified and given a recommendation:

Not every file in a spool is queued work. A Debian or Ubuntu postfix keeps a chroot in /var/spool/postfix (etc/hosts, certificates, libraries), anacron keeps timestamps in /var/spool/anacron, and on the Red Hat family /var/spool/mail holds delivered mailboxes. So a spool row says one of three things about its files:

• queued: <size> in <n> file(s): files in the queue of a layout discover knows. This is work in flight, and the choice is yours: drain it on the source before cutover, or carry it by naming the spool with --copy. queued: none found means the queue directories were read and held no file no package owns. If part of the spool was not read it says queued: not known instead, and if some queued files were counted and another part was not read it says queued: at least <size> in <n> file(s). A queue directory that is a symbolic link, or a file of that name, counts as not read: discover does not follow links, so what is behind one is not counted as this spool's queue even when the link points into a tree that is walked.
• A description of the rest, such as service files, not queued mail or not a queue: delivered mailboxes, with its size and file count. Draining the queue does not change this figure.
• not classified: <size> in <n> file(s): the spool's layout is not one discover knows, so it does not say whether the files are queued work. Look at the directory.

The layouts told apart are postfix (queue: active, bounce, corrupt, defer, deferred, hold, incoming, maildrop, saved, trace), exim4 and exim (queue: input, msglog), cups (queue: the c<number> and d<number>-<number> files directly in the directory), and two that hold no queue, anacron and mail. A directory with one of the first three names that holds none of its queue directories is not classified. Every other name under /var/spool is not classified. In --json a classified spool has layout, queued_files and queued_bytes; a spool with no layout key is not classified, and its two counts are 0 without meaning that nothing is queued.

A spool that could not be read is listed at the top as NOT READ with the reason, and in --json under not_read. That covers a spool directory the user cannot list, a spool that is a link into a tree discover does not walk, a link at or inside a queue directory of a known layout wherever it points, and any --ssh source, where /var/spool is not collected. It is not a report that the spool is empty. Unreadable directories outside /var/spool are only counted, in skipped.

LOCATION                                       SIZE   FILES  RECOMMENDED
/var/lib/mysql                              8.0 KiB       1  db-export (mysql)
/home/appuser/uploads                       4.0 KiB       1  byte-copy
/srv/www/site                                  14 B       1  byte-copy

plan

Takes the paths you select with --copy (repeatable) and writes four artifacts to -o <out>:

A path that matches a database signature is always routed to the DB runbook, never to a raw copy — even if you pass it to --copy.

Attestation

The manifest proves the copy is byte-identical. After copying, on the target:

sha256sum -c vrtmv-data-manifest.sha256   # every line must say OK

Optionally, point the tool at a destination tree and it re-hashes it for you, marking each file match / differ / missing / unreadable in the attestation JSON (it still never performs the copy or the dump):

vrtmv data plan --root /mnt/vm --copy /srv/www --verify-dest /mnt/target/srv/www -o out

The exit code is the answer. It is 0 only when at least one file was compared, every compared file matches, and every --copy path was compared. A missing or differing file, a file that could not be read, or a --copy path that was not compared (a database that is exported, a path that is not on the source or has no files there) exits 1, and stderr has one line with the counts. That holds when another path in the same run matches in full: the line then reads NOT CHECKED in full and names the path that was left out, so a mistyped --copy is not a pass. An empty destination is the case this is for: every file is missing and the exit is 1. A destination that cannot be listed is refused before anything is compared, and is not reported as empty. A link below the destination is not followed: a file whose path passes through a link is differ, as it is for verify --data-dir. The plan is still written when the exit is 1.

--verify-dest names the directory that holds the content of the one --copy path, as above. For a --copy path that is one file, give the copy of the file, or the directory the copy is in, where it is looked up by its name: --copy /etc/motd --verify-dest /mnt/target/etc/motd and --verify-dest /mnt/target/etc say the same. Given the copy itself, only its bytes are compared: its name is yours to choose and is not checked, so --verify-dest /backup/motd.copy verifies too. Given the directory, the name is what finds it. A file is accepted as the destination of one --copy path that is a file on the source and is refused for anything else. With --dest-dir /data/vol the copy of /srv/www is at /data/vol/srv/www, so that is the directory to give. With more than one --copy every path is looked up directly under the same directory, which is rarely what was copied. Two --copy paths that are files of the same name (/srv/a/x.txt and /srv/b/x.txt) would both be x.txt in that directory, where one copy would pass for both, so that is refused: the error names both files, nothing is compared and nothing is written. The same collision between a file and a file inside a --copy directory, or between files of two --copy directories, is not detected. Verify one path per run, or use verify --data-dir, which takes a directory per path.

--verify-dest refuses a destination that has a directory above it which an account other than root and the one running vrtmv can write to. That account can rename what is in the directory, so it can put another tree where the destination was while it is being read. The error names the directory, its owner and its mode, nothing is compared and nothing is written. What counts: an owner that is neither root nor you, or a group or world write bit. A sticky directory such as /tmp is passed as a parent, because there only the owner of an entry can rename it. A link on the way is followed, and its owner and the directories it leads through are judged the same. A directory above the destination that cannot be looked at is refused too: it is not taken as one nobody else can write to. The destination itself is not judged. A copy that kept its owners belongs to whoever owned the source, so its owner says nothing about the copy; that owner can still change the files in it. A group write bit is counted whoever is in the group, so a directory with mode 775 and a group only you are in is refused.

--allow-shared-dest compares anyway. The comparison's line on stderr is followed by one that starts SHARED and repeats what the refusal would have said, and the attestation records allow_shared_dest: true and the same text as verify_dest_shared. Neither field is written by a run without the flag, and verify_dest_shared is written only when there was something to let through.

A local directory as the destination

By default the copy script sends each path to $TARGET:<path> over SSH. With --dest-dir <dir> it fills a directory on the machine the script runs on instead, which is the form to use when the data is going into a volume for a container built by migrate --format container:

vrtmv data plan --root /mnt/vm --copy /srv/www --copy /var/lib/app --dest-dir /data/vol -o out
SOURCE_ROOT=/mnt/vm bash out/vrtmv-data-copy.sh
podman run -v /data/vol/srv/www:/srv/www -v /data/vol/var/lib/app:/var/lib/app ...

data plan does not create or read the directory. The script does the work when you run it, with SOURCE_ROOT set to the source root (/ on the source host itself). It places each path at <dir><path>, keeps owners and modes, and compares the copy with vrtmv-data-manifest.sha256, which it expects beside it. Run it as root unless every file belongs to the user running it.

The script refuses a destination that is a symbolic link or that already holds anything, and leaves it untouched.

It also refuses a destination that an account other than root and the one running the script can write to, and one that has such a directory above it, by the rule --verify-dest uses, here with the destination itself judged as well. It names the directory, its owner and its mode. The script stages in <dir>/.vrtmv-partial, and an account that can write to <dir> can rename that directory away and put a link in its place while the script copies: measured as root, a path was copied through such a link into a directory that account could not write to. A directory above the destination that the script cannot look at is refused, not taken as safe. The script looks twice: before it makes the destination, which may be absent then, and again once the destination is there. In a sticky directory such as /var/tmp any account can make a name that is absent, so a destination another account made between the two looks is refused at the second, and so is one that is gone again by then. A destination the script makes has no group or world write bit whatever the umask is. The check is made by the script when it runs, on the machine it runs on; data plan does not look at the directory. A plan made with --allow-shared-dest writes a script that goes on instead. It says what it went past before it writes anything and again on its last line, and the attestation records allow_shared_dest: true. The flag does not make the copy safe: the script has no defence against the swap other than refusing the directory.

If any file cannot be placed, it names each one, removes what it copied and exits 1. That covers a file that is gone from the source or changed since the plan, a file added to the source since the plan, and an owner or mode that was not kept. The last line of a failed run says what is in the destination after the cleanup: nothing, or the names that are still there, which can be names the script did not make. It checks the owner, group, mode and type of every entry and the sha256 of every file. It does not check timestamps, ACLs, extended attributes or hard links.

A database is never copied by this script. Its section in vrtmv-db-runbook.md names the directory (<dir><path>) that the import has to fill. With --source-offline, on an engine where a cold copy is allowed, the database's directory is copied by the script like any other path, under the same warning lines as in the SSH form.

With --dest-dir, data plan refuses a set of --copy paths in which one is under another or one is given twice, and names the pair. Give the outer path only. The script holds the same rule on its own: it makes every directory between the destination and a copy itself, and stops at a symbolic link or at a path that an earlier copy already placed, so that no copy is made through a link from the source to somewhere outside the destination.

A file whose name contains a newline cannot be placed. The script fails by name and leaves nothing.

One plan fills one empty directory, once. A Kubernetes claim is not a destination yet.

Named podman volumes as the destination

With --dest-volume <path>=<volume>, given once for every --copy path that is copied, the script creates one named podman volume per path and fills it. Use it when the container is to mount volumes by name and not directories:

vrtmv data plan --root /mnt/vm --copy /srv/www --copy /var/lib/app \
  --dest-volume /srv/www=www --dest-volume /var/lib/app=appdata -o out
SOURCE_ROOT=/mnt/vm HELPER_IMAGE=localhost/myapp:1 bash out/vrtmv-data-copy.sh
podman run -v www:/srv/www -v appdata:/var/lib/app ...

data plan creates nothing and does not run podman. The script does, when you run it, on a machine that has podman and the source root. A volume is not a directory the script can copy into from outside, so the copy is made inside a helper container: the source path is mounted read-only, the volume read-write, the container has no network, and the copy and the checks are done by the image's own cp, find and sha256sum. HELPER_IMAGE names that image. It must already be in podman's local storage, since the script pulls nothing, and it needs /bin/sh, GNU coreutils and GNU findutils. The image you built from the migration's Containerfile is the usual choice. The script looks the name up once and runs the image by its id.

The top of each volume is the content of its path: /srv/www/index.html is index.html in the volume www, and the volume's top directory gets the owner and mode of /srv/www. The script makes the same checks as the local-directory form, on the same list: owner, group, mode and type of every entry, and the sha256 of every file.

Each volume must not exist: the script creates the volumes it fills and refuses, before it creates any, if one of them is already there. If any file cannot be placed it names each one, removes every volume it created in that run, the ones that filled correctly included, and exits 1. A helper container that ends without its last line (it was killed, or the image has no shell) is such a failure. A helper image without GNU find is one too.

Run the script as root unless every file belongs to the user running it. Rootless podman maps only its own user and its subordinate ids into the container, so a file owned by anyone else cannot be given its owner, and the script fails on it by name.

A database the plan exports gets no volume from the script. --dest-volume is still accepted for its path, and its section in vrtmv-db-runbook.md then names the volume the import has to fill. --dest-volume is refused for a path that is not one of the --copy paths, for a path the plan found missing or could not measure, for a --copy path that is a single file, when a copied path has no volume, when two paths name one volume, and when one copied path is under another. It cannot be combined with --dest-dir.

To check a volume afterwards, see verify --data-volume.

Kubernetes claims as the destination

With --dest-claim <path>=<claim>, given once for every --copy path that is copied, the script packs what an init container in the cluster will fetch:

vrtmv data plan --root /mnt/source --copy /srv/www --copy /var/lib/app \
  --dest-claim /srv/www=www-data --dest-claim /var/lib/app=app-data -o out
SOURCE_ROOT=/mnt/source bash out/vrtmv-data-copy.sh

data plan writes out/claims/<claim>.sha256 for each copied path: the sha256 and name of every file under it, as sha256sum -c reads them. The script checks every file of the source against that list, then packs out/claims/<claim>.tar with GNU tar, keeping numeric owners, modes, links and empty directories. Set ARCHIVE_DIR to pack somewhere else; the list is copied beside the archive. If the source has changed since the plan, or holds a file the plan does not list, the script names the path, removes every archive it wrote in that run and exits 1.

Nothing is uploaded and nothing in a cluster is touched. Upload each .tar and .sha256 to one place in an S3-compatible bucket, then give that place to vrtmv deploy --kind container --data-source-url, which renders the claim and the init container that fills and checks it. The claim's name is the name of its two files, so use the same name in both commands.

A database the plan exports gets no archive. --dest-claim is still accepted for its path, and its section in vrtmv-db-runbook.md then names the claim. --dest-claim is refused for the same cases as --dest-volume, and for a file name with a control character in it. A claim's name is lowercase letters, digits and -. It cannot be combined with --dest-dir or --dest-volume.

Not covered: a volume that already exists (one you created with a driver or mount options of your own), timestamps, ACLs, extended attributes, and rootless podman with files of more than one owner. A Kubernetes claim is not a destination yet.

Databases

Detected by their vendor-documented on-disk data directory (and, for relocated dirs, a distinctive marker file). The runbook emits the supported dump/restore for each:

Every command is a real vendor-documented procedure, cited in the emitted runbook — Vrtmv surfaces the steps; the operator runs them.