runtimeterror/content/posts/publish-silverbullet-notes-quartz/index.md at 374b9d522ce4659badf44e8f4e6a35e36e885c68

john/runtimeterror

Fork 0

mirror of https://github.com/jbowdre/runtimeterror.git synced 2024-11-26 17:02:17 +00:00

John Bowdre 374b9d522c update draft

2024-10-26 17:15:51 -05:00

20 KiB

Raw Blame History

title

date

draft

description

featured

toc

Plugging in the Git plug

SilverBullet can be extended through the use of plugs, and installing the Git plug should make it easy to sync my SilverBullet content to a private GitHub repo.

But I should probably initialize my space (the SilverBullet equivalent of a vault/notebook/graph) as a git repo first.

Recall from my setup notes that I'm mounting a folder named ./space into my SilverBullet container at /space. I'll need to turn that into a git repo so I SSH to my Docker host, move into the folder containing my SilverBullet space, and initialize the repo:

cd /opt/silverbullet/space # [tl! .cmd:1]
git init .

I'll connect this local git repo to a private GitHub repo, but I'll need to use a Personal Access Token (PAT) for the git interactions since the Git plug running inside the SilverBullet container won't have access to my SSH private key. So I create a new PAT, scope it only to my new private repo (jbowdre/spaaace), and grant it the contents: write permission there. I can then use the PAT when I set up the remote and push my first commit:

git remote add origin https://github_pat_[...]@github.com/jbowdre/spaaace.git # [tl! .cmd:3]
git add .
git commit -m "initial commit"
git push --set-upstream origin main

This stores the authentication token directly inside the local git configuration (in /opt/silverbullet/space/.git/config) so that git operations performed within the container will be automatically authenticated.

Now that my repo is ready, I can go ahead and install and configure the Git plug. I do that by logging into my SilverBullet instance on the web (https://silverbullet.tailnet-name.ts.net), pressing [Ctrl] + / to bring up the command palette, typing/selecting Plugs: Add, and pasting in the URI for the Git plug: github:silverbulletmd/silverbullet-git/git.plug.js.

The docs say that I can add the following to my SETTINGS file to enable automatic syncing:

git:
  autoCommitMinutes: 5
  autoSync: true

But that doesn't actually seem to work for some reason (at least for me). That's okay, though, because I can easily add a keyboard shortcut ([Ctrl] + [Alt] + .) to quickly sync on-demand:

shortcuts:
  - command: "{[Git: Sync]}"
    key: "Ctrl-Alt-."

{{% notice note "Brace for it..." %}} Note that the command target for the shortcut is wrapped with a square bracket wrapped with a curly brace ({[ ]}). It won't work if you do a Go-template-style double-curly-braces ({{ }}).

Ask me how I know (and how long it took me to find my mistake!). {{% /notice %}}

I'll use [Ctrl] + / to get the command pallette again and run System: Reload to activate my change, and then simply pressing [Ctrl] + [Alt] + . will trigger a git pull + git commit + git push (as needed) sequence.

That takes care of getting my SilverBullet content into GitHub. Now let's see how it gets published.

Setting up Quartz

"Installing" Quartz is pretty straight forward thanks to the instructions on the Quartz website. I just ran these commands on my laptop:

git clone https://github.com/jackyzha0/quartz.git # [tl! .cmd:3]
cd quartz
npm i
npx quartz create

By default, Quartz expects my Obsidian content to be in the content/ directory (and there's a placeholder file there for now). I'll replace that with my spaaace repo for testing but also add that path to the .gitignore file to ensure I don't accidentally commit my private notes:

rm -rf content/ # [tl! .cmd:2]
git clone git@github.com:jbowdre/spaaace.git content
echo "content" >> .gitignore

From there I can move on to configuring Quartz. The documentation has helpful information on some of the configuration options so I'm just going to highlight the changes that are particularly important to this specific setup.

In the plugins: section of quartz.config.ts, I enable the ExplicitPublish filter plugin to tell Quartz to only render pages with publish: true in the frontmatter:

  plugins: {
    filters: [
      Plugin.RemoveDrafts(),
      Plugin.ExplicitPublish(), // [tl! ++]
    ],
    [...]
  }

That will allow me very granular control over which posts are published (and which remain private), but the Private Pages Quartz documentation page warns that the ExplicitPublish plugin only filters out Markdown files. All other files (images, PDFs, plain TXTs) will still be processed and served publicly. I don't intend to include screenshots or other media with these short code-heavy notes so I scroll back up in the quartz.config.ts file and add a little regex to the ignorePatterns section:

  configuration: {
    ignorePatterns: [
      "private",
      "templates",
      "**/!(*.md)" // [tl! ++]
    ],
    [...]
  }

That will avoid processing any non-Markdown files.

The rest of the Quartz setup follows the documentation, including the steps to connect my (public) GitHub repository.

Before publishing, I can check my work by generating and serving the Quartz content locally:

npx quartz build --serve # [tl! .cmd ** .nocopy:1,13]

 Quartz v4.4.0

Cleaned output directory `public` in 29ms
Found 198 input files from `content` in 24ms
Parsed 198 Markdown files in 4s
Filtered out 198 files in 398μs
⠋ Emitting output files
Warning: you seem to be missing an `index.md` home page file at the root of your `content` folder. This may cause errors when deploying. # [tl! ** ~~]
Emitted 8 files to `public` in 44ms
Done processing 0 files in 103ms
Started a Quartz server listening at http://localhost:8080
hint: exit with ctrl+c

Oops! Remember how the ExplicitPublish plugin will only process notes with publish: true set in the frontmatter? Since I just imported the notes from my (public) Obsidian vault into SilverBullet none of them have that attribute set yet. (But hey, this is a great way to test that my filters work!)

Let me run through the notes I want to be public and update them accordingly...

---
title: Trigger remote workflow with GitHub Actions
tags: [github]
publish: true # [tl! ++]
---
...

And then I'll try again:

npx quartz build --serve # [tl! .cmd ** .nocopy:1,11]

 Quartz v4.4.0

Cleaned output directory `public` in 6ms
Found 198 input files from `content` in 32ms
Parsed 198 Markdown files in 4s # [tl! **:2]
Filtered out 123 files in 404μs
Emitted 130 files to `public` in 497ms
Done processing 198 files in 4s
Started a Quartz server listening at http://localhost:8080 # [tl! **]
hint: exit with ctrl+c

That's more like it!

But serving my notes from my laptop is only so useful. Let's keep going and see what it takes to publish them on the World Wide Web!

Publish publicly

I've previously written about my GitHub Actions workflow for publishing my Gemini capsule, and I'm going to reuse a lot of the same ideas here. I'll create a workflow that performs the steps needed to render the HTML to the public/ directory, establishes a Tailscale tunnel to my server, and transfers the rendered content there. Those static files will then be served with Caddy, taking advantage of its automatic HTTPS abilities.

Server prep

The setup on the server is pretty simple. I just create a directory to hold the files, and make sure it's owned by the deploy user:

sudo mkdir /opt/notes # [tl! .cmd:1]
sudo chown -R deploy:deploy /opt/notes

I'll also go ahead and update my Caddyfile based on the Quartz documentation, but I won't reload Caddy just yet (I'll wait until I have some content to serve):

notes.runtimeterror.dev {
    bind 192.0.2.1    # replace with server's public interface address
    root * /opt/notes/public
    try_files {path} {path}.html {path}/ =404
    file_server
    encode gzip

    handle_errors {
        rewrite * /{err.status_code}.html
        file_server
    }
}

Tailscale prep

The full details of how I configured Tailscale to support this deploy-from-github-actions use case are available in another post so I won't repeat the explanation here. But these are the items I added to my Tailscale ACL to create a set of tags (one for the GitHub runner, one for the server it will deploy to), allow SSH traffic from the runner to the server, and configure Tailscale SSH to let the runner log in to the server as the deploy user:

{
  "tagOwners": {
    "tag:gh-bld":      ["group:admins"], // github builder
    "tag:gh-srv":      ["group:admins"], // server it can deploy to
  },
  "acls": [
    {
      // github runner can talk to the deployment target
      "action": "accept",
      "users":  ["tag:gh-bld"],
      "ports": [
        "tag:gh-srv:22"
      ],
    }
  ],
  "ssh": [
    {
      // runner can SSH to the server as the 'deploy' user
      "action": "accept",
      "src": ["tag:gh-bld"],
      "dst": ["tag:gh-srv"],
      "users": ["deploy"],
    }
  ],
}

Workin' on a workflow

With the prep out of the way, I'm ready to start on my deployment workflow.

My .github/workflows/deploy.yaml starts simply with just setting some defaults, and it configures the workflow to run on pushes to the default branch (v4), repository_dispatch events, and workflow_dispatch events (manual executions).

# torchlight! {"lineNumbers":true}
name: Deploy Notes

# run on changes to default (v4) branch, repository_dispatch events, and manual executions
on:
  push:
    branches:
      - v4
  repository_dispatch:
  workflow_dispatch:

concurrency: # prevent concurrent deploys doing strange things
  group: deploy
  cancel-in-progress: false

# Default to bash
defaults:
  run:
    shell: bash

The deploy job then starts with checking out the repo where the job is running and the private repo holding my SilverBullet space, which gets cloned to the content/ directory.

To be able to fetch the private jbowdre/spaaace repo, I'll needed to generate another PAT scoped to that repo. This one only needs read access (no write) to the contents of the repo. The PAT and the repo path get stored as repository secrets.

# torchlight! {"lineNumbers": true, "lineNumbersStart": 20}
jobs:
  deploy:
    name: Build and deploy Quartz site
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Quartz
        uses: actions/checkout@v4
      - name: Checkout notes
        uses: actions/checkout@v4
        with:
          repository: ${{ secrets.SPAAACE_REPO }}
          token: ${{ secrets.SPAAACE_REPO_PAT }}
          path: content

I can then move on to installing Node and building the Quartz site:

# torchlight! {"lineNumbers":true, "lineNumbersStart": 33}
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: 20
      - name: Build Quartz
        run: |
          npm ci
          npx quartz build

I use the Tailscale GitHub Action to connect the ephemeral GitHub runner to my tailnet, and apply that ACL tag that grants it SSH access to my web server (and nothing else).

I've also stored that web server's SSH public key as a repository secret, and I make sure that gets added to the runner's ~/.ssh/known_hosts file so that it can connect without being prompted to verify the host keys.

Finally, I use rsync to copy the public/ directory (with all the rendered HTML content) to /opt/notes/ on the server.

# torchlight! {"lineNumbers":true, "lineNumbersStart":41}
      - name: Connect to Tailscale
        uses: tailscale/github-action@v2
        with:
          oauth-client-id: ${{ secrets.TS_API_CLIENT_ID }}
          oauth-secret: ${{ secrets.TS_API_CLIENT_SECRET }}
          tags: ${{ secrets.TS_TAG }}
      - name: Configure SSH known hosts
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.SSH_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
          chmod 644 ~/.ssh/known_hosts          
      - name: Deploy Quartz
        run: |
          rsync -avz --delete -e ssh public/ deploy@${{ secrets.QUARTZ_HOST }}:${{ secrets.QUARTZ_CONTENT_PATH }}

After making sure that I've added all the required repository secrets, I can commit and push my code and it should trigger the deployment...

git add . # [tl! .cmd:2]
git commit -m "deployment test"
git push

And I can log back onto my server and confirm that the content is there:

ls -l /opt/notes/public/ # [tl! .cmd .nocopy:1,17]
drwxr-xr-x    - deploy 29 Sep 19:04 ChromeOS
drwxr-xr-x    - deploy 29 Sep 19:04 CICD
drwxr-xr-x    - deploy 29 Sep 19:04 Containers
drwxr-xr-x    - deploy 29 Sep 19:04 Development
drwxr-xr-x    - deploy 29 Sep 19:04 Linux
drwxr-xr-x    - deploy 29 Sep 19:04 Saltstack
drwxr-xr-x    - deploy 29 Sep 19:04 static
drwxr-xr-x    - deploy 29 Sep 19:04 tags
drwxr-xr-x    - deploy 29 Sep 19:04 VMware
drwxr-xr-x    - deploy 29 Sep 19:04 Windows
.rw-r--r-- 3.9k deploy 29 Sep 19:04 404.html
.rw-r--r--  30k deploy 29 Sep 19:04 index.css
.rw-r--r--  25k deploy 29 Sep 19:04 index.html
.rw-r--r-- 4.8k deploy 29 Sep 19:04 index.xml
.rw-r--r--  62k deploy 29 Sep 19:04 postscript.js
.rw-r--r--  903 deploy 29 Sep 19:04 prescript.js
.rw-r--r--  11k deploy 29 Sep 19:04 sitemap.xml

Now that I've got some content I can reload Caddy:

sudo caddy reload -c /etc/caddy/Caddyfile # [tl! .cmd .nocopy:1,2]
2024/09/29 19:11:17.705 INFO    using config from file  {"file": "/etc/caddy/Caddyfile"}
2024/09/29 19:11:17.715 INFO    adapted config to JSON  {"adapter": "caddyfile"}

And check to see if the site is up:

Nice, my notes are online!

Trigger workflow

The last piece of this puzzle is to trigger the deployment workflow whenever my SilverBullet notes get synced to that private repo. Fortunately I have a note that describes how to do that.

I'll set up yet another GitHub PAT, this one scoped to the jbowdre/notes public repo with permissions to write to the repository contents. Then I just need a workflow in the private jbowdre/spaaace repo to make a POST to https://api.github.com/repos/jbowdre/notes/dispatches whenever a Markdown file is created/updated.

Here's .github/workflows/trigger.yaml:

# torchlight! {"lineNumbers":true}
name: Trigger Quartz Build

on:
  push:
    paths:
      - "**.md"

defaults:
    run:
      shell: bash

jobs:
  publish:
    name: Trigger
    runs-on: ubuntu-latest
    steps:
      - name: Remote trigger
        run: |
          curl -X POST \
            -H "Authorization: token ${{ secrets.NOTES_REPO_PAT }}" \
            -H "Accept: application/vnd.github.v3+json" \
            https://api.github.com/repos/${{ secrets.NOTES_REPO }}/dispatches \
            -d '{"event_type": "remote-trigger"}'

Once I commit and push this, any future changes to a markdown file will tell the GitHub API to kick off the remote workflow in the public repo.

Conclusion

And that's it! I can now write and publish short notes from anywhere thanks to the SilverBullet web app, Quartz, and some GitHub Actions shenanigans. This was a little more work to set up on the front end but my new publishing workflow couldn't be simpler: just write a note and hit [Ctrl] + [Alt] + . to sync the change to the private GitHub repo and kick off the deployment.

Now I don't have an excuse to keep sitting on this backlog of quick notes I've been meaning to share...

20 KiB Raw Blame History