The problem

Build dependencies, alternatively called packages or libraries, can disappear from dependency servers. Dependency servers can go offline, even if only temporarily. This can leave projects in a state where they can not build.

The corporate solution

The easiest way for companies to solve this is to use their own dependency server and configure their build processes to pull dependencies from and publish dependencies to their private dependency server. That dependency server can then serve as a backup for the public dependency servers. It also becomes easier to use dependencies that were developed in-house. Example solutions are Artifactory and AWS CodeArtifact .

Other things to consider

Source code and documentation for libraries may also be considered important artifacts for the development process, depending on your specific situation. Documentation may disappear from the web and source code that used to be available might become closed off. I have also seen cases where applications and libraries that we started using many years back were sold, renamed or rewritten. As a result online documentation and sources were too recent for us, as well as difficult to find because names and websites had changed. Consider making backups of documentation and source code for libraries, and installers and documentation for software.

A hobby project solution

A less ideal solution is storing the dependencies offline. To be able to use backups of build dependencies we would have to set the build system to an offline mode where we point it to a directory. A better way might be to use the build system's own caching system and copy them there. That is easy to do with Java's Maven build system, but may need configuration and could be more difficult for other build systems such as Gradle .

One thing to consider is if one wants to make backups of entire build system cache directories, or just backup dependencies used in individual project's build steps.

For a hobby project written in Java I am trying out the Gradle task below. It should run on every build. It copies all dependencies that the build system, Gradle, can find. The copied files can be added to version control, or to some kind of backup system if that is preferred.

I am considering changing that code to use the information resolved by Gradle to look up the matching entries in the Gradle cache directory and backing up those directories instead. That way it would be easy to insert them back in to the cache and let Gradle use them.

task backupResolvedLibs() {
    doFirst {
        var outDirPath = Paths.get(project.projectDir.toPath().toAbsolutePath().toString(), 
                                   'build_dependencies'+File.separator)
        String outDir = outDirPath.toString()
        Files.createDirectories(outDirPath)
        configurations.resolvableImpl.resolvedConfiguration.resolvedArtifacts.each {
            var source = it.getFile().toPath().toAbsolutePath()
            var target = Paths.get(outDir, it.getFile().getName()).toAbsolutePath()
            println source
            println target
            if (it.getFile().getName().contains("SNAPSHOT") || !Files.exists(target)) {
                Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING)
            }
        }
    }
}

We need to make sure that the task runs regularly. In Gradle that means adding it to another task, a commonly run task. It also needs to be run after running the build step, to make sure that the dependencies have actually been resolved. The lines we need to add to that Gradle task look something like this:

dependsOn 'clean'
dependsOn 'build'
dependsOn 'backupResolvedLibs'
tasks.findByName('build').mustRunAfter 'clean'
tasks.findByName('backupResolvedLibs').mustRunAfter 'build'

One nice way to add things to build steps is to use a shell script that fails when any step fails . The shell script can call the 'backupResolvedLibs' Gradle task after the build has succeeded.