Synchronize with Neocities us­ing Git and curl

Created: Tue Jan 29 04:53:14 CET 2019

Last mod­i­fied: Fri Feb 1 03:45:04 CET 2019


The is­sue with the of­fi­cial Neocities client

Simply put, it takes too much time for big web­sites.

So, you reg­is­tered to Neocities, you have a big web­site, thou­sands of files, and you used neocities push . to syn­chro­nize them.

If you’re run­ning Linux, you prob­a­bly did gem install neocities or something like that be­fore you could run neocities push .. (We will call this tool neocities.rb for the sake of clar­ity.)

Then you pushed your files for a sec­ond time af­ter chang­ing two or three of them and… the tool will do as you say and for each file (in a thousand), (1) will ver­ify that it changed up­stream (2) up­load it if so.

Approximately 1 or 2 sec­ond per file. What the hell. Clearly neocities.rb is (extremely) use­ful for small web­sites only.

The workaround

We’ll use a ver­sion­ing sys­tem to check which files changed since the last time we synced with Neocities.

Retrieve your API key, on Neocities or di­rectly in ~/.config/neocities/config if you al­ready tried neocities.rb and put it in­side of a file named .secret at the root of your web­site

At the root of your web­site: git init and echo .secret >> .gitignore (so that .secret will never be up­loaded).

Now, each time a few files are al­tered, you can run the fol­low­ing script with cd my-website && ./build.sh. (Don’t for­get to chmod u+x build.sh.)

#!/usr/bin/env bash
set -e

Here is a func­tion to ab­stract curl away.

function upload() {
  curl -H "Authorization: Bearer $(tr -d '\n' < .secret)" \
       -F "$1=@$1" "https://neocities.org/api/upload" >/dev/null 2>&1
}

git status --porcelain out­puts a bunch of lines of the for­mat XX this-file-changed.txt. Loop over it and it­er­a­tively call upload on the sec­ond col­umn.

for f in $(git status --porcelain | awk '{ print $2 }')
do
  printf "Uploading $f..."
  upload ./$f
  echo " Done!"
done

Validate every ed­its at once so they won’t show up next time in git-status(1)s out­put.

git add . && git commit -m "bump"

Directories are cre­ated as needed by Neocities’ back­end. Meaning that ./assets/css/style.css will be kept in sync with file at https://YOURWEBSITE/assets/css/style.css with­out ex­tra code.

source code