The bxpublish program generates a copy of a directory structure with certain files omitted. It can be used to provide a publication directory as a staging area for uploading a Blox project. Using bxpublish, it is possible to prevent staging file which have been recreated without change in content. In this way, traffic for upload might be reduced. It then suffices to compare the modification dates of the files in the publication directory and those on the server.
bxpublish reads its specification about which files to include and which not from configuration files called publish.txt. There are two kinds of instruction lines in a publish.txt file:
The directory patterns as well as the name patterns are interpreted as regular expressions. Space characters inside of them will be part of the pattern. An example publish.txt file is:
- notneeded\.txt + included\.tar\.gz dir(ectory)?: - .*\.ext
Each path group line pertains to the following definition lines, until another path group line is read. Because of this behaviour path group lines partition the definition lines into groups.
Comments are started by a hashmark # and extend till the end of the line. Comments are filtered out before any other processing of the line. Because of the whitespace permitted at the end of definition lines and path group lines, whitespace immediately before the hash mark will be ignored. Comments might in turn contain any number of further hash marks. There is no way to escape a hash mark, so it cannot be used as part of a path group or definition pattern.
The bxpublish program will first read all publish.txt files, in order to decide about each and every file in the source directory whether it needs inclusion or not. This results in an inventory of files which shall be published. The ongoing operation of bxpublish consists of three steps:
Files will be copied when the content of the resp. source file differs from the target file's content. There are several conditions which can be met when deciding about a file part of the inventory whether it needs copying or not:
When the destination exists as a directory, the file will be skipped. In this case, since the origin exists as a file, no files below the destination will be part of the inventory, s.t. the destination will be eligible for pruning right after the file-pruning step and will be deleted in the directory pruning step. bxpublish should thus be ran again; an appropriate message will be output. Only in dry run mode the destination might survive.
In case the destination does not exist or the size of the destination differs from the size of the source, the file will be copied over using rsync -t.
When the origin and the destination exist and are of equal size, their content is compared in order to decide about the copying operation.
In the beginning of its operation, bxpublish looks for a hash generator. Currently considered are sha1sum and md5sum. The choice can be overridden manually by the environmental variable HASHGEN. The value none stands for the absence of any hash generator. The hash generator is expected to produce output starting with a file's hash when being called as $ <HASHGENERATOR> <FILE>.
With a hash generator present, it will be used to compare the contents of the origin and the target files, and when the file needs copying, rsync -t will be used on it.
When no hash generator could be found, the -c option to rsync will be used to hand over the decision to rsync, thus rsync -tc will effectively be employed.
Source files which are content-identical to their destination counterpart are to be skipped. In case of the absence of a hash generator, this condition however cannot be detected, as it is indistinguishable from the situation of differing content with equal size, where copying is required; so copying cannot be skipped. Under such circumstances all source files will be copied over unconditionally, possibly with hand-over of the content question to rsync. The different reasons for performing a copy operation are still iterated to provide fine-grained information to the user at increased verbosity levels. Notice that rsync might skip the file on its own, when rsync -tc detects using its built-in hash algorithm that there are no differences present. However, only in the highest verbosity level the user will be able to notice.
To find the publish.txt files, bxpublish looks for directories beneath the source directory, checking for each directory found whether it contains such a file. The list of directory pathes is sorted prior to this examination.
While reading the publish.txt files, a catalogue of rules is generated, one for each definition line. Each such rule has four components:
There is a default exclusion rule defined to not publish the publish.txt files. This rule is rooted at the source directory, its path pattern matches any path and its name pattern applies precisely to publish.txt.
For each file within the source directory it is ascertainted for each rule if the file is covered by the given rule:
Definition lines at the top of a publish.txt file, before any path group line, will create rules with the empty string as their path pattern. Such rules thus cover files directly within the root path of the publish.txt file.
When multiple rules cover a file, the one defined last is decisive. When no rule covers a file in question, the file will be included by default. Because only the directory pathes are sorted during publish.txt file discovery, it is guaranteed that each publish.txt file in a certain directory is read earlier than publish.txt files in subfolders of the resp. directory.