initial work at migrating content to new site

This commit is contained in:
John Bowdre 2023-08-14 17:05:16 -05:00
parent b1b6384e85
commit de3c5a9331
220 changed files with 4469 additions and 3 deletions

2
.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
.hugo_build.lock

9
.gitmodules vendored Normal file
View file

@ -0,0 +1,9 @@
[submodule "themes/risotto"]
path = themes/risotto
url = https://github.com/joeroe/risotto.git
[submodule "themes/hugo-notice"]
path = themes/hugo-notice
url = https://github.com/martignoni/hugo-notice.git
[submodule "themes/hugo-cloak-email"]
path = themes/hugo-cloak-email
url = https://github.com/martignoni/hugo-cloak-email.git

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2014 Steve Francia
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View file

@ -1,3 +0,0 @@
baseURL = 'http://example.org/'
languageCode = 'en-us'
title = 'My New Hugo Site'

View file

@ -0,0 +1,26 @@
baseURL = "https://runtimeterror.dev"
theme = [ "hugo-cloak-email", "hugo-notice", "risotto"]
title = "runtimeterror"
author = "ops"
copyright = "© 2023 [runtimeterror](https://runtimeterror.dev)"
paginate = 10
languageCode = "en"
DefaultContentLanguage = "en"
enableInlineShortcodes = true
# Automatically add content sections to main menu
sectionPagesMenu = "main"
[outputs]
home = ["HTML", "RSS", "JSON"]
[permalinks]
posts = ":filename"
[services]
[services.instagram]
disableInlineCSS = true
[services.twitter]
disableInlineCSS = true

View file

@ -0,0 +1,27 @@
timeout = 30000
enableInlineShortcodes = true
[taxonomies]
category = "categories"
tag = "tags"
series = "series"
[privacy]
[privacy.vimeo]
disabled = false
simple = true
[privacy.twitter]
disabled = false
enableDNT = true
simple = true
disableInlineCSS = true
[privacy.instagram]
disabled = false
simple = true
[privacy.youtube]
disabled = false
privacyEnhanced = true

View file

@ -0,0 +1,11 @@
# For hugo >= 0.60.0, enable inline HTML
[goldmark.renderer]
unsafe = true
# Table of contents
# Add toc = true to content front matter to enable
[tableOfContents]
startLevel = 2
endLevel = 3
ordered = true

View file

@ -0,0 +1,5 @@
[[main]]
identifier = "about"
name = "About"
url = "/about/"
weight = 10

View file

@ -0,0 +1,27 @@
noindex = false
usePageBundles = true
[theme]
palette = "runtimeterror"
# Sidebar: about/bio
[about]
title = "runtimeterror"
description = "Better living through less-bad code."
logo = "images/broken-computer.svg"
# Sidebar: social links
# Available icon sets:
# * FontAwesome 6 <https://fontawesome.com/> ('fa-brands', 'fa-normal', or 'fa-solid' for brands)
# * Academicons <https://jpswalsh.github.io/academicons> ('ai ai-')
[[socialLinks]]
icon = "fa-brands fa-github"
title = "GitHub"
url = "https://github.com/jbowdre"
[[socialLinks]]
icon = "fa-solid fa-envelope"
title = "Email"
url = "mailto:ops@runtimeterror.dev"

3
content/_index.md Normal file
View file

@ -0,0 +1,3 @@
+++
author = "ops"
+++

25
content/about.md Normal file
View file

@ -0,0 +1,25 @@
+++
title = "About"
description = "Hugo, the world's fastest framework for building websites"
date = "2019-02-28"
aliases = ["about-us", "about-hugo", "contact"]
author = "Hugo Authors"
+++
Written in Go, Hugo is an open source static site generator available under the [Apache Licence 2.0.](https://github.com/gohugoio/hugo/blob/master/LICENSE) Hugo supports TOML, YAML and JSON data file types, Markdown and HTML content files and uses shortcodes to add rich content. Other notable features are taxonomies, multilingual mode, image processing, custom output formats, HTML/CSS/JS minification and support for Sass SCSS workflows.
Hugo makes use of a variety of open source projects including:
* https://github.com/yuin/goldmark
* https://github.com/alecthomas/chroma
* https://github.com/muesli/smartcrop
* https://github.com/spf13/cobra
* https://github.com/spf13/viper
Hugo is ideal for blogs, corporate websites, creative portfolios, online magazines, single page applications or even a website with thousands of pages.
Hugo is for people who want to hand code their own website without worrying about setting up complicated runtimes, dependencies and databases.
Websites built with Hugo are extremely fast, secure and can be deployed anywhere including, AWS, GitHub Pages, Heroku, Netlify and any other hosting provider.
Learn more and contribute on [GitHub](https://github.com/gohugoio).

5
content/archives.md Normal file
View file

@ -0,0 +1,5 @@
---
date: 2019-05-28
type: section
layout: "archives"
---

View file

@ -0,0 +1,7 @@
---
title: 'Our Difference'
button: 'About us'
weight: 2
---
Lorem ipsum dolor sit amet, et essent mediocritatem quo, choro volumus oporteat an mei. Ipsum dolor sit amet, et essent mediocritatem quo.

View file

@ -0,0 +1,3 @@
---
headless: true
---

7
content/homepage/work.md Normal file
View file

@ -0,0 +1,7 @@
---
title: 'We Help Business Grow'
button: 'Our Work'
weight: 1
---
Lorem ipsum dolor sit amet, et essent mediocritatem quo, choro volumus oporteat an mei. Numquam dolores mel eu, mea docendi omittantur et, mea ea duis erat. Elit melius cu ius. Per ex novum tantas putant, ei his nullam aliquam apeirian. Aeterno quaestio constituto sea an, no eum intellegat assueverit.

Binary file not shown.

After

Width:  |  Height:  |  Size: 552 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 198 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

View file

@ -0,0 +1,96 @@
---
date: "2020-09-14T08:34:30Z"
thumbnail: qDTXt1jp3.png
featureImage: qDTXt1jp3.png
usePageBundles: true
tags:
- linux
- chromeos
- crostini
- 3dprinting
title: 3D Modeling and Printing on Chrome OS
---
I've got an Ender 3 Pro 3D printer, a Raspberry Pi 4, and a Pixel Slate. I can't interface directly with the printer over USB from the Slate (plus having to be physically connected to things is like so lame) so I installed [Octoprint on the Raspberry Pi](https://github.com/guysoft/OctoPi) and connected that to the printer's USB interface. This gave me a pretty web interface for controlling the printer - but it's only accessible over the local network. I also installed [The Spaghetti Detective](https://www.thespaghettidetective.com/) to allow secure remote control of the printer, with the added bonus of using AI magic and a cheap camera to detect and abort failing prints.
That's a pretty sweet setup, but I still needed a way to convert STL 3D models into GCODE files which the printer can actually understand. And what if I want to create my own designs?
Enter "Crostini," Chrome OS's [Linux (Beta) feature](https://chromium.googlesource.com/chromiumos/docs/+/master/containers_and_vms.md). It consists of a hardened Linux VM named `termina` which runs (by default) a Debian Buster LXD container named `penguin` (though you can spin up just about any container for which you can find an [image](https://us.images.linuxcontainers.org/)) and some fancy plumbing to let Chrome OS and Linux interact in specific clearly-defined ways. It's a brilliant balance between offering the flexibility of Linux while preserving Chrome OS's industry-leading security posture.
![Neofetch in the Crostini terminal](lhTnVwCO3.png)
There are plenty of great guides (like [this one](https://www.computerworld.com/article/3314739/linux-apps-on-chrome-os-an-easy-to-follow-guide.html)) on how to get started with Linux on Chrome OS so I won't rehash those steps here.
One additional step you will probably want to take is make sure that your Chromebook is configured to enable hyperthreading, as it may have [hyperthreading disabled by default](https://support.google.com/chromebook/answer/9340236). Just plug `chrome://flags/#scheduler-configuration` into Chrome's address bar, set it to `Enables Hyper-Threading on relevant CPUs`, and then click the button to restart your Chromebook. You'll thank me later.
![Enabling hyperthreading](LHax6lAwh.png)
### The Software
I settled on using [FreeCAD](https://www.freecadweb.org/) for parametric modeling and [Ultimaker Cura](https://ultimaker.com/software/ultimaker-cura) for my GCODE slicer, but unfortunately getting them working cleanly wasn't entirely straightforward.
#### FreeCAD
Installing FreeCAD is as easy as:
```shell
$ sudo apt update
$ sudo apt install freecad
```
But launching `/usr/bin/freecad` caused me some weird graphical defects which rendered the application unusable. I found that I needed to pass the `LIBGL_DRI3_DISABLE=1` environment variable to eliminate these glitches:
```shell
$ env 'LIBGL_DRI3_DISABLE=1' /usr/bin/freecad &
```
To avoid having to type that every time I wished to launch the app, I inserted this line at the bottom of my `~/.bashrc` file:
```shell
alias freecad="env 'LIBGL_DRI3_DISABLE=1' /usr/bin/freecad &"
```
To be able to start FreeCAD from the Chrome OS launcher with that environment variable intact, edit it into the `Exec` line of the `/usr/share/applications/freecad.desktop` file:
```shell
$ sudo vi /usr/share/applications/freecad.desktop
[Desktop Entry]
Version=1.0
Name=FreeCAD
Name[de]=FreeCAD
Comment=Feature based Parametric Modeler
Comment[de]=Feature-basierter parametrischer Modellierer
GenericName=CAD Application
GenericName[de]=CAD-Anwendung
Exec=env LIBGL_DRI3_DISABLE=1 /usr/bin/freecad %F
Path=/usr/lib/freecad
Terminal=false
Type=Application
Icon=freecad
Categories=Graphics;Science;Engineering
StartupNotify=true
GenericName[de_DE]=Feature-basierter parametrischer Modellierer
Comment[de_DE]=Feature-basierter parametrischer Modellierer
MimeType=application/x-extension-fcstd
```
That's it! Get on with your 3D-modeling bad self.
![FreeCAD](qDTXt1jp3.png)
Now that you've got a model, be sure to [export it as an STL mesh](https://wiki.freecadweb.org/Export_to_STL_or_OBJ) so you can import it into your slicer.
#### Ultimaker Cura
Cura isn't available from the default repos so you'll need to download the AppImage from https://github.com/Ultimaker/Cura/releases/tag/4.7.1. You can do this in Chrome and then use the built-in File app to move the file into your 'My Files > Linux Files' directory. Feel free to put it in a subfolder if you want to keep things organized - I stash all my AppImages in `~/Applications/`.
To be able to actually execute the AppImage you'll need to adjust the permissions with 'chmod +x':
```shell
$ chmod +x ~/Applications/Ultimaker_Cura-4.7.1.AppImage
```
You can then start up the app by calling the file directly:
```shell
$ ~/Applications/Ultimaker_Cura-4.7.1.AppImage &
```
AppImages don't automatically appear in the Chrome OS launcher so you'll need to create its `.desktop` file. You can do this manually if you want, but I found it a lot easier to leverage `menulibre`:
```shell
$ sudo apt update && sudo apt install menulibre
$ menulibre
```
Just plug in the relevant details (you can grab the appropriate icon [here](https://github.com/Ultimaker/Cura/blob/master/icons/cura-128.png)), hit the filing cabinet Save icon, and you should then be able to search for Cura from the Chrome OS launcher.
![Using menulibre to create the launcher shortcut](VTISYOKHO.png)
![Ultimaker Cura](f8nRJcyI6.png)
From there, just import the STL mesh, configure the appropriate settings, slice, and save the resulting GCODE. You can then just upload the GCODE straight to The Spaghetti Detective and kick off the print.
![Successful print, designed and sliced on Chrome OS!](2g57odtq2.jpeg)
Nice!

Binary file not shown.

After

Width:  |  Height:  |  Size: 180 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 664 KiB

6
content/post/_index.md Normal file
View file

@ -0,0 +1,6 @@
+++
aliases = ["posts", "articles", "blog", "showcase", "docs"]
title = "Posts"
author = "Hugo Authors"
tags = ["index"]
+++

Binary file not shown.

After

Width:  |  Height:  |  Size: 211 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 228 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 347 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 218 KiB

View file

@ -0,0 +1,72 @@
---
series: Tips
date: "2020-09-24T08:34:30Z"
thumbnail: fmLDUWjia.png
usePageBundles: true
tags:
- chrome
title: Abusing Chrome's Custom Search Engines for Fun and Profit
---
Do you (like me) find yourself frequently searching for information within the same websites over and over? Wouldn't it be great if you could just type your query into your browser's address bar (AKA the Chrome Omnibox) and go straight to the results you need? Well you totally can - and probably already *are* for certain sites which have inserted themselves as search engines.
### The basics
Point your browser to `chrome://settings/searchEngines` to see which sites are registered as Custom Search Engines:
![Custom search engines list](RuIrsHDqC.png)
Each of these search engine entries has three parts: a name ("Search engine"), a Keyword, and a Query URL. The "Search engine" title is just what will appear in the Omnibox when the search engine gets triggered, the Keyword is what you'll type in the Omnibox to trigger it, and the Query URL tells Chrome how to handle the search. All you have to do is type the keyword, hit your Tab key to activate the search, input your query, and hit Enter:
![Using a custom search engine](o_o7rt4pA.gif)
For sites which register themselves automatically, the keyword is often set to something like `domain.tld` so it might make sense to assign it as something shorter or more descriptive.
The Query URL is basically just what appears in the address bar when you search the site directly, with `%s` placed where your query text would normally go. You can view these details for a given search entry by tapping the three-dot menu button and selecting "Edit", and you can manually create new entries by hitting that big friendly "Add" button:
![Editing a search engine](fmLDUWjia.png)
By searching the site directly, you might find that it supports additional search filters which get appended to the URL:
![Discovering search filters](iHsYd7lbw.png)
You can add those filters to the Query URL to further customize your Custom Search Engine:
![Adding filters to a custom search](EBkQTGmNb.png)
I spend a lot of my free time helping out on Google's support forums as a part of their [Product Experts program](https://productexperts.withgoogle.com/what-it-is), and I often need to quickly look up a Help Center article or previous forum discussion to assist users. I created a set of Custom Search Engines to make that easier:
![Google Help Center search engines](630ix7uVw.png)
![Pixel Buds Help search](V3qLmfi50.png)
------
### Creating search where there is none
Even if the site doesn't have a built-in native search, you can leverage Google's `sitesearch` operator to create one. I often want to look up a Linux command's `man` page, so I use this Query URL to search https://www.man7.org/linux/man-pages/:
```
http://google.com/search?q=%s&sitesearch=man7.org%2Flinux%2Fman-pages
```
![man search](EkmgtRYN4.png)
![Searching man](YKADY8YQR.gif)
------
### Speak foreign to me
This works for pretty much any site which parses the URL to render certain content. I use this for getting words/phrases instantly translated:
![Google Translate search](ELly_F6x6.png)
![Translating German with search!](1LDP5zxCU.gif)
------
### Shorter shortcuts
Your Query URL doesn't even need to include a query at all! You can use the Custom Search Engines as a sort of hyper-fast shortcut to pages you visit frequently. If I create a new entry with the Keyword `searchax` and `abusing-chromes-custom-search-engines-for-fun-and-profit` as the query URL, I can quickly open to this page by typing `searchax[tab][enter]`:
![Custom search shortener](YilNCaHil.png)
I use that trick pretty regularly for getting back to vCenter appliance management interfaces without having to type out the full FQDN and port number and all that.
------
### Scratchpad hack
You can do some other creative stuff too, like speedily accessing a temporary scratchpad for quickly jotting down notes, complete with spellcheck! Just drop this into the Query URL field:
```
data:text/html;charset=utf-8, <title>Scratchpad</title><style>body {padding: 5%; font-size: 1.5em; font-family: Arial; }"></style><link rel="shortcut icon" href="https://ssl.gstatic.com/docs/documents/images/kix-favicon6.ico"/><body OnLoad='document.body.focus();' contenteditable spellcheck="true" >
```
And give it a nice short keyword - like the single letter 's':
![My own scratchpad!](h6dUCApdV.gif)
------
With just a bit of tweaking, you can really supercharge Chrome's Omnibox capabilities. Let me know if you come across any other clever uses for this!

Binary file not shown.

After

Width:  |  Height:  |  Size: 334 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 121 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 384 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

View file

@ -0,0 +1,184 @@
---
series: Projects
date: "2021-05-27T08:34:30Z"
thumbnail: HRRpFOKuN.png
usePageBundles: true
tags:
- docker
- vmware
- containers
- networking
- security
title: AdGuard Home in Docker on Photon OS
---
I was recently introduced to [AdGuard Home](https://adguard.com/en/adguard-home/overview.html) by way of its very slick [Home Assistant Add-On](https://github.com/hassio-addons/addon-adguard-home/blob/main/adguard/DOCS.md). Compared to the relatively-complicated [Pi-hole](https://pi-hole.net/) setup that I had implemented several months back, AdGuard Home was *much* simpler to deploy (particularly since I basically just had to click the "Install" button from the Home Assistant add-ons manage). It also has a more modern UI with options arranged more logically (to me, at least), and it just feels easier to use overall. It worked great for a time... until my Home Assistant instance crashed, taking down AdGuard Home (and my internet access) with it. Maybe bundling these services isn't the best move.
I'd like to use AdGuard Home, but the system it runs on needs to be rock-solid. With that in mind, I thought it might be fun to instead run AdGuard Home in a Docker container on a VM running VMware's container-optimized [Photon OS](https://github.com/vmware/photon), primarily because I want an excuse to play more with Docker and Photon (but also the thing I just mentioned about stability). So here's what it took to get that running.
### Deploy Photon
First, up: getting Photon. There are a variety of delivery formats available [here](https://github.com/vmware/photon/wiki/Downloading-Photon-OS), and I opted for the HW13 OVA version. I copied that download URL:
```
https://packages.vmware.com/photon/4.0/GA/ova/photon-hw13-uefi-4.0-1526e30ba0.ova
```
Then I went into vCenter, hit the **Deploy OVF Template** option, and pasted in the URL:
![Deploying the OVA straight from the internet](Es90-kFW9.png)
This lets me skip the kind of tedious "download file from internet and then upload file to vCenter" dance, and I can then proceed to click through the rest of the deployment options.
![Ready to deploy](rCpaTbPX5.png)
Once the VM is created, I power it on and hop into the web console. The default root username is `changeme`, and I'll of course be forced to change that the first time I log in.
### Configure Networking
My next step was to configure a static IP address by creating `/etc/systemd/network/10-static-en.network` and entering the following contents:
```conf
[Match]
Name=eth0
[Network]
Address=192.168.1.2/24
Gateway=192.168.1.1
DNS=192.168.1.5
```
By the way, that `192.168.1.5` address is my Windows DC/DNS server that I use for [my homelab environment](/vmware-home-lab-on-intel-nuc-9#basic-infrastructure). That's the DNS server that's configured on my Google Wifi router, and it will continue to handle resolution for local addresses.
I also disabled DHCP by setting `DHCP=no` in `/etc/systemd/network/99-dhcp-en.network`:
```conf
[Match]
Name=e*
[Network]
DHCP=no
IPv6AcceptRA=no
```
I set the required permissions on my new network configuration file with `chmod 644 /etc/systemd/network/10-static-en.network` and then restarted `networkd` with `systemctl restart systemd-networkd`.
I then ran `networkctl` a couple of times until the `eth0` interface went fully green, and did an `ip a` to confirm that the address had been applied.
![Verifying networking](qOw7Ysj3O.png)
One last little bit of housekeeping is to change the hostname with `hostnamectl set-hostname adguard` and then reboot for good measure. I can then log in via SSH to continue the setup.
![SSH login](NOyfgjjUy.png)
Now that I'm in, I run `tdnf update` to make sure the VM is fully up to date.
### Install docker-compose
Photon OS ships with Docker preinstalled, but I need to install `docker-compose` on my own to simplify container deployment. Per the [install instructions](https://docs.docker.com/compose/install/#install-compose), I run:
```shell
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
```
And then verify that it works:
```shell
root@adguard [ ~]# docker-compose --version
docker-compose version 1.29.2, build 5becea4c
```
I'll also want to enable and start Docker:
```shell
systemctl enable docker
systemctl start docker
```
### Disable DNSStubListener
By default, the `resolved` daemon is listening on `127.0.0.53:53` and will prevent docker from binding to that port. Fortunately it's [pretty easy](https://github.com/pi-hole/docker-pi-hole#installing-on-ubuntu) to disable the `DNSStubListener` and free up the port:
```shell
sed -r -i.orig 's/#?DNSStubListener=yes/DNSStubListener=no/g' /etc/systemd/resolved.conf
rm /etc/resolv.conf && ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
systemctl restart systemd-resolved
```
### Deploy AdGuard Home container
Okay, now for the fun part.
I create a directory for AdGuard to live in, and then create a `docker-compose.yaml` therein:
```shell
mkdir ~/adguard
cd ~/adguard
vi docker-compose.yaml
```
And I define the container:
```yaml
version: "3"
services:
adguard:
container_name: adguard
restart: unless-stopped
image: adguard/adguardhome:latest
ports:
- "53:53/tcp"
- "53:53/udp"
- "67:67/udp"
- "68:68/tcp"
- "68:68/udp"
- "80:80/tcp"
- "443:443/tcp"
- "853:853/tcp"
- "3000:3000/tcp"
volumes:
- './workdir:/opt/adguardhome/work'
- './confdir:/opt/adguardhome/conf'
cap_add:
- NET_ADMIN
```
Then I can fire it up with `docker-compose up --detach`:
```shell
root@adguard [ ~/adguard ]# docker-compose up --detach
Creating network "adguard_default" with the default driver
Pulling adguard (adguard/adguardhome:latest)...
latest: Pulling from adguard/adguardhome
339de151aab4: Pull complete
4db4be09618a: Pull complete
7e918e810e4e: Pull complete
bfad96428d01: Pull complete
Digest: sha256:de7d791b814560663fe95f9812fca2d6dd9d6507e4b1b29926cc7b4a08a676ad
Status: Downloaded newer image for adguard/adguardhome:latest
Creating adguard ... done
```
### Post-deploy configuration
Next, I point a web browser to `http://adguard.lab.bowdre.net:3000` to perform the initial (minimal) setup:
![Initial config screen](UHvtv1DrT.png)
Once that's done, I can log in to the dashboard at `http://adguard.lab.bowdre.net/login.html`:
![Login page](34xD8tbli.png)
AdGuard Home ships with pretty sensible defaults so there's not really a huge need to actually do a lot of configuration. Any changes that I *do* do will be saved in `~/adguard/confdir/AdGuardHome.yaml` so they will be preserved across container changes.
### Getting requests to AdGuard Home
Normally, you'd tell your Wifi router what DNS server you want to use, and it would relay that information to the connected DHCP clients. Google Wifi is a bit funny, in that it wants to function as a DNS proxy for the network. When you configure a custom DNS server for Google Wifi, it still tells the DHCP clients to send the requests to the router, and the router then forwards the queries on to the configured DNS server.
I already have Google Wifi set up to use my Windows DC (at `192.168.1.5`) for DNS. That lets me easily access systems on my internal `lab.bowdre.net` domain without having to manually configure DNS, and the DC forwards resolution requests it can't handle on to the upstream (internet) DNS servers.
To easily insert my AdGuard Home instance into the flow, I pop in to my Windows DC and configure the AdGuard Home address (`192.168.1.2`) as the primary DNS forwarder. The DC will continue to handle internal resolutions, and anything it can't handle will now get passed up the chain to AdGuard Home. And this also gives me a bit of a failsafe, in that queries will fail back to the previously-configured upstream DNS if AdGuard Home doesn't respond within a few seconds.
![Setting AdGuard Home as a forwarder](bw09OXG7f.png)
It's working!
![Requests!](HRRpFOKuN.png)
### Caveat
Chaining my DNS configurations in this way (router -> DC -> AdGuard Home -> internet) does have a bit of a limitation, in that all queries will appear to come from the Windows server:
![Only client](OtPGufxlP.png)
I won't be able to do any per-client filtering as a result, but honestly I'm okay with that as I already use the "Pause Internet" option in Google Wifi to block outbound traffic from certain devices anyway. And using the Windows DNS as an intermediary makes it significantly quicker and easier to switch things up if I run into problems later; changing the forwarder here takes effect instantly rather than having to manually update all of my clients or wait for DHCP to distribute the change.
I have worked around this in the past by [bypassing Google Wifi's DHCP](https://www.mbreviews.com/pi-hole-google-wifi-raspberry-pi/) but I think it was actually more trouble than it was worth to me.
### One last thing...
I'm putting a lot of responsibility on both of these VMs, my Windows DC and my new AdGuard Home instance. If they aren't up, I won't have internet access, and that would be a shame. I already have my ESXi host configured to automatically start up when power is (re)applied, so I also adjust the VM Startup/Shutdown Configuration so that AdGuard Home will automatically boot after ESXi is loaded, followed closely by the Windows DC (and the rest of my virtualized infrastructure):
![Auto Start-up Options](clE6OVmjp.png)
So there you have it. Simple DNS-based ad-blocking running on a minimal container-optimized VM that *should* be more stable than the add-on tacked on to my Home Assistant instance. Enjoy!

Binary file not shown.

After

Width:  |  Height:  |  Size: 339 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 730 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 229 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 401 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 478 KiB

View file

@ -0,0 +1,179 @@
---
series: Projects
date: "2020-11-24T08:34:30Z"
lastmod: "2021-03-12"
thumbnail: Ki7jo65t3.png
usePageBundles: true
tags:
- android
- automation
- tasker
- vpn
title: Auto-connect to ProtonVPN on untrusted WiFi with Tasker [Update!]
---
*[Update 2021-03-12] This solution recently stopped working for me. While looking for a fix, I found that OpenVPN had published [some notes](https://openvpn.net/faq/how-do-i-use-tasker-with-openvpn-connect-for-android/) on controlling the [official OpenVPN Connect app](https://play.google.com/store/apps/details?id=net.openvpn.openvpn) from Tasker. Jump to the [Update](#update) below to learn how I adapted my setup with this new knowledge.*
I recently shared how I use [Tasker and Home Assistant to keep my phone from charging past 80%](/safeguard-your-androids-battery-with-tasker-home-assistant). Today, I'm going to share the setup I use to automatically connect my phone to a VPN on networks I *don't* control.
![Tasker + OpenVPN](Ki7jo65t3.png)
### Background
Android has an option to [set a VPN as Always-On](https://support.google.com/android/answer/9089766#always-on_VPN) so for maximum security I could just use that. I'm not *overly* concerned (yet?) with my internet traffic being intercepted upstream of my ISP, though, and often need to connect to other devices on my home network without passing through a VPN (or introducing split-tunnel complexity). But I do want to be sure that my traffic is protected whenever I'm connected to a WiFi network controlled by someone else.
I've recently started using [ProtonVPN](https://protonvpn.com/) in conjunction with my paid ProtonMail account so these instructions are tailored to that particular VPN provider. I'm paying for the ProtonVPN Plus subscription but these instructions should also work for the [free tier](https://protonvpn.com/free-vpn) as well. (And this should work for any VPN which provides an OpenVPN config file - you'll just have to find that on your own.)
ProtonVPN does provide a quite excellent [Android app](https://play.google.com/store/apps/details?id=ch.protonvpn.android) but I couldn't find a way to automate it without root. (If your phone is rooted, you should be able to use a Tasker shell to run `cmd statusbar click-tile ch.protonvpn.android/com.protonvpn.android.components.QuickTileService` and avoid needing to use OpenVPN at all.)
### The apps
You'll need a few apps to make this work:
- [Tasker](https://play.google.com/store/apps/details?id=net.dinglisch.android.taskerm)
- [OpenVPN for Android](https://play.google.com/store/apps/details?id=de.blinkt.openvpn)
- [OpenVpn Tasker Plugin](https://play.google.com/store/apps/details?id=com.ffrog8.openVpnTaskerPlugin)
It's important to use the [open-source](https://github.com/schwabe/ics-openvpn) 'OpenVPN for Android' app by Arne Schwabe rather than the 'OpenVPN Connect' app as <s>the latter doesn't work with the Tasker plugin</s> that's what I used when I originally wrote this guide.
### OpenVPN config file
You can find instructions for configuring the OpenVPN client to work with ProtonVPN [here](https://protonvpn.com/support/android-vpn-setup/) but I'll go ahead and hit the highlights. You'll probably want to go ahead and do all this from your phone so you don't have to fuss with transferring files around, but hey, *you do you*.
1. Log in to your ProtonVPN account (or sign up for a new free one) at https://account.protonvpn.com/login.
2. Use the panel on the left side to navigate to **[Downloads > OpenVPN configuration files](https://account.protonvpn.com/downloads#openvpn-configuration-files)**.
3. Select the **Android** platform and **UDP** as the protocol, unless you have a [particular reason to use TCP](https://protonvpn.com/support/udp-tcp/#:~:text=When%20to%20use%20UDP%20vs.%20TCP).
4. Select and download the desired config file:
- **Secure Core configs** utilize the [Secure Core](https://protonvpn.com/support/secure-core-vpn/) feature which connects you to a VPN node in your target country by way of a Proton-owned-and-managed server in privacy-friendly Iceland, Sweden, or Switzerland
- **Country configs** connect to a random VPN node in your target country
- **Standard server configs** let you choose the specific VPN node to use
- **Free server configs** connect you to one of the VPN nodes available in the free tier
![Client config download page](vdIG0jHmk.png)
Feel free to download more than one if you'd like to have different profiles available within the OpenVPN app.
ProtonVPN automatically generates a set of user credentials to use with a third-party VPN client so that you don't have to share your personal creds. You'll want to make a note of that randomly-generated username and password so you can plug them in to the OpenVPN app later. You can find the details at **[Account > OpenVPN / IKEv2 username](https://account.protonvpn.com/account#openvpn)**.
**Now that you've got the profile file, skip on down to [The Update](#update) to import it into OpenVPN Connect.**
### Configuring OpenVPN for Android
Now what you've got the config file(s) and your client credentials, it's time to actually configure that client.
![OpenVPN connection list](9WdA6HRch.png)
1. Launch the OpenVPN for Android app and tap the little 'downvote-in-a-box' "Import" icon.
2. Browse to wherever you saved the `.ovpn` config files and select the one you'd like to use.
3. You can rename if it you'd like but I feel that `us.protonvpn.com.udp` is pretty self-explanatory and will do just fine to distinguish between my profiles. Tap the check mark at the top-right or the floppy icon at the bottom right to confirm the import.
4. Now tap the pencil icon next to the new entry to edit its settings, and paste in the OpenVPN username and password where appropriate. Use your phone's back button/gesture to save the config and return to the list.
5. Repeat for any other configurations you'd like to import. We'll only use one for this particular Tasker profile but you might come up with different needs for different scenarios.
6. And finally, tap on the config name to test the connection. The OpenVPN Log window will appear, and you want the line at the top to (eventually) display something like `Connected: SUCCESS`.
Success!
I don't like to have a bunch of persistent notification icons hanging around (and Android already shows a persistent status icon when a VPN connection is active). If you're like me, long-press the OpenVPN notification and tap the gear icon. Then tap on the **Connection statistics** category and activate the **Minimized** slider. The notification will still appear, but it will collapse to the bottom of your notification stack and you won't get bugged by the icon.
![Notification settings](WWuHwVvrk.png)
### Tasker profiles
Open up Tasker and get ready to automate! We're going to wind up with at least two new Tasker profiles so (depending on how many you already have) you might want to create a new project by long-pressing the Home icon at the bottom-left of the screen and selecting the **Add** option. I chose to group all my VPN-related profiles in a project named (oh-so-creatively) "VPN". Totally your call though.
Let's start with a profile to track whether or not we're connected to one of our preferred/trusted WiFi networks:
#### Trusted WiFi
1. Tap the '+' sign to create a new profile, and add a new **State > Net > Wifi Connected** context. This profile will become active whenever your phone connects to WiFi.
2. Tap the magnifying glass next to the **SSID** field, which will pop up a list of all detected nearby network identifiers. Tap to select whichever network(s) you'd like to be considered "safe". You can also manually enter the SSID names, separating multiple options with a `/` (ex, `FBI Surveillance Van/TellMyWifiLoveHer/Pretty fly for a WiFi`). Or, for more security, identify the networks based on the MACs instead of the SSIDs - just be sure to capture the MACs for any extenders or mesh nodes too!
3. Once you've got your networks added, tap the back button to move *forward* to the next task (Ah, Android!): configuring the *action* which will occur when the context is satisfied.
4. Tap the **New Task** option and then tap the check mark to skip giving it a name (no need).
5. Hit the '+' button to add an action and select **Variables > Variable Set**.
6. For **Name**, enter `%TRUSTED_WIFI` (all caps to make it a "public" variable), and for the **To** field just enter `1`.
7. Hit back to save the action, and back again to save the profile.
8. Back at the profile list, long-press on the **Variable Set...** action and then select **Add Exit Task**.
9. We want to un-set the variable when no longer connected to a trusted WiFi network so add a new **Variables > Variable Clear** action and set the name to `%TRUSTED_WIFI`.
10. And back back out to admire your handiwork. Here's a recap of the profile:
```
Profile: Trusted Wifi
State: Wifi Connected [ SSID:FBI Surveillance Van/TellMyWifiLoveHer/Pretty fly for a WiFi MAC:* IP:* Active:Any ]
Enter: Anon
A1: Variable Set [ Name:%TRUSTED_WIFI To:1 Recurse Variables:Off Do Maths:Off Append:Off Max Rounding Digits:0 ]
Exit: Anon
A1: Variable Clear [ Name:%TRUSTED_WIFI Pattern Matching:Off Local Variables Only:Off Clear All Variables:Off ]
```
Onward!
#### VPN on Strange WiFi
This profile will kick in if the phone connects to a WiFi network which isn't on the "approved" list - when the `%TRUSTED_WIFI` variable is not set.
1. It starts out the same way by creating a new profile with the **State > Net > Wifi Connected** context but this time don't add any network names to the list.
2. For the action, select **Plugin > OpenVpn Tasker Plugin**, tap the pencil icon to edit the configuration, and select your VPN profile from the list under **Connect using profile**
3. Back at the Action Edit screen, tap the checkbox next to **If** and enter the variable name `%TRUSTED_WIFI`. Tap the '~' button to change the condition operator to **Isn't Set**. So while this profile will activate every time you connect to WiFi, the action which connects to the VPN will only fire if the WiFi isn't a trusted network.
4. Back out to the profile list and add a new Exit Task.
5. Add another **Plugin > OpenVpn Tasker Plugin** task and this time configure it to **Disconnect VPN**.
To recap:
```
Profile: VPN on Strange Wifi
State: Wifi Connected [ SSID:* MAC:* IP:* Active:Any ]
Enter: Anon
A1: OpenVPN [ Configuration:Connect (us.protonvpn.com.udp) Timeout (Seconds):0 ] If [ %TRUSTED_WIFI !Set ]
Exit: Anon
A1: OpenVPN [ Configuration:Disconnect Timeout (Seconds):0 ]
```
### Conclusion
Give it a try - the VPN should automatically activate the next time you connect to a network that's not on your list. If you find that it's not working correctly, you might try adding a short 3-5 second **Task > Wait** action before the connect/disconnect actions just to give a brief cooldown between state changes.
### Epilogue: working with Google's VPN
My Google Pixel 5 has a neat option at **Settings > Network & internet > Wi-Fi > Wi-Fi preferences > Connect to public networks** which will automatically connect the phone to known-decent public WiFi networks and automatically tunnel the connection through a Google VPN. It doesn't provide quite as much privacy as ProtonVPN, of course, but it's enough to keep my traffic safe from prying eyes on those public networks, and the auto-connection option really comes in handy sometimes. Of course, my Tasker setup would see that I'm connected to an unknown network and try to connect to ProtonVPN at the same time the phone was trying to connect to the Google VPN. That wasn't ideal.
I came up with a workaround to treat any network with the Google VPN as "trusted" as long as that VPN was active. I inserted a 10-second Wait before the Connect and Disconnect actions to give the VPN time to stand up, and added two new profiles to detect the Google VPN connection and disconnection.
#### Google VPN On
This one uses an **Event > System > Logcat Entry**. The first time you try to use that you'll be prompted to use adb to grant Tasker the READ_LOGS permission but the app actually does a great job of walking you through that setup. We'll watch the `Vpn` component and filter for `Established by com.google.android.apps.gcs on tun0`, and then set the `%TRUSTED_WIFI` variable:
```
Profile: Google VPN On
Event: Logcat Entry [ Output Variables:* Component:Vpn Filter:Established by com.google.android.apps.gcs on tun0 Grep Filter (Check Help):Off ]
Enter: Anon
A1: Variable Set [ Name:%TRUSTED_WIFI To:1 Recurse Variables:Off Do Maths:Off Append:Off Max Rounding Digits:3 ]
```
#### Google VPN Off
This one is pretty much the same but the opposite:
```
Profile: Google VPN Off
Event: Logcat Entry [ Output Variables:* Component:Vpn Filter:setting state=DISCONNECTED, reason=agentDisconnect Grep Filter (Check Help):Off ]
Enter: Anon
A1: Variable Clear [ Name:%TRUSTED_WIFI Pattern Matching:Off Local Variables Only:Off Clear All Variables:Off ]
```
### Update
#### OpenVPN Connect app configuration
After installing and launching the official [OpenVPN Connect app](https://play.google.com/store/apps/details?id=net.openvpn.openvpn), tap the "+" button at the bottom right to create a new profile. Swipe over to the "File" tab and import the `*.ovpn` file you downloaded from ProtonVPN. Paste in the username, tick the "Save password" box, and paste in the password as well. I also chose to rename the profile to something a little bit more memorable - you'll need this name later. From there, hit the "Add" button and then go ahead and tap on your profile to test the connection.
![Creating a profile in OpenVPN Connect](KjGOX8Yiv.png)
#### Tasker profiles
Go ahead and create the [Trusted Wifi profile](#trusted-wifi) as described above.
The condition for the [VPN on Strange Wifi profile](#vpn-on-strange-wifi) will be the same, but the task will be different. This time, add a **System > Send Intent** action. You'll need to enter the following details, leaving the other fields blank/default:
```
Action: net.openvpn.openvpn.CONNECT
Cat: None
Extra: net.openvpn.openvpn.AUTOSTART_PROFILE_NAME:PC us.protonvpn.com.udp (replace with your profile name)
Extra: net.openvpn.openvpn.AUTOCONNECT:true
Extra: net.openvpn.openvpn.APP_SECTION:PC
Package: net.openvpn.openvpn
Class: net.openvpn.unified.MainActivity
Target: Activity
If: %TRUSTED_WIFI !Set
```
The Exit Task to disconnect from the VPN uses a similar intent:
```
Action: net.openvpn.openvpn.DISCONNECT
Cat: None
Extra: net.openvpn.openvpn.STOP:true
Package: net.openvpn.openvpn
Class: net.openvpn.unified.MainActivity
Target: Activity
```
All set! You can pop back up to the [Epilogue](#epilogue-working-with-googles-vpn) section to continue tweaking to avoid conflicts with Google's auto-connect VPN if you'd like.

Binary file not shown.

After

Width:  |  Height:  |  Size: 236 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

View file

@ -0,0 +1,231 @@
---
series: Projects
date: "2018-09-26T08:34:30Z"
lastmod: "2022-03-06"
thumbnail: i0UKdXleC.png
usePageBundles: true
tags:
- docker
- linux
- cloud
- gcp
- security
title: BitWarden password manager self-hosted on free Google Cloud instance
---
![Bitwarden login](i0UKdXleC.png)
A friend mentioned the [BitWarden](https://bitwarden.com/) password manager to me yesterday and I had to confess that I'd never heard of it. I started researching it and was impressed by what I found: it's free, [open-source](https://github.com/bitwarden), feature-packed, fully cross-platform (with Windows/Linux/MacOS desktop clients, Android/iOS mobile apps, and browser extensions for Chrome/Firefox/Opera/Safari/Edge/etc), and even offers a self-hosted option.
I wanted to try out the self-hosted setup, and I discovered that the [official distribution](https://help.bitwarden.com/article/install-on-premise/) works beautifully on an `n1-standard-1` 1-vCPU Google Compute Engine instance - but that would cost me an estimated $25/mo to run after my free Google Cloud Platform trial runs out. And I can't really scale that instance down further because the embedded database won't start with less than 2GB of RAM.
I then came across [this comment](https://www.reddit.com/r/Bitwarden/comments/8vmwwe/best_place_to_self_host_bitwarden/e1p2f71/) on Reddit which discussed in somewhat-vague terms the steps required to get BitWarden to run on the [free](https://cloud.google.com/free/docs/always-free-usage-limits#compute_name) `e2-micro` instance, and also introduced me to the community-built [vaultwarden](https://github.com/dani-garcia/vaultwarden) project which is specifically designed to run a BW-compatible server on resource-constrained hardware. So here are the steps I wound up taking to get this up and running.
{{% notice info "bitwarden_rs -> vaultwarden"%}}
When I originally wrote this post back in September 2018, the containerized BitWarden solution was called `bitwarden_rs`. The project [has since been renamed](https://github.com/dani-garcia/vaultwarden/discussions/1642) to `vaultwarden`, and I've since moved to the hosted version of BitWarden. I have attempted to update this article to account for the change but have not personally tested this lately. Good luck, dear reader!
{{% /notice %}}
### Spin up a VM
*Easier said than done, but head over to https://console.cloud.google.com/ and fumble through:*
1. Creating a new project (or just add an instance to an existing one).
2. Creating a new Compute Engine instance, selecting `e2-micro` for the Machine Type and ticking the *Allow HTTPS traffic* box.
3. *(Optional)* Editing the instance to add an ssh-key for easier remote access.
### Configure Dynamic DNS
*Because we're cheap and don't want to pay for a static IP.*
1. Log in to the [Google Domain admin portal](https://domains.google.com/registrar) and [create a new Dynamic DNS record](https://domains.google.com/registrar). This will provide a username and password specific for that record.
2. Log in to the GCE instance and run `sudo apt-get update` followed by `sudo apt-get install ddclient`. Part of the install process prompts you to configure things... just accept the defaults and move on.
3. Edit the `ddclient` config file to look like this, substituting the username, password, and FDQN from Google Domains:
```shell
$ sudo vi /etc/ddclient.conf
# Configuration file for ddclient generated by debconf
#
# /etc/ddclient.conf
protocol=googledomains,
ssl=yes,
syslog=yes,
use=web,
server=domains.google.com,
login='[USERNAME]',
password='[PASSWORD]',
[FQDN]
```
4. `sudo vi /etc/default/ddclient` and make sure that `run_daemon="true"`:
```shell
# Configuration for ddclient scripts
# generated from debconf on Sat Sep 8 21:58:02 UTC 2018
#
# /etc/default/ddclient
# Set to "true" if ddclient should be run every time DHCP client ('dhclient'
# from package isc-dhcp-client) updates the systems IP address.
run_dhclient="false"
# Set to "true" if ddclient should be run every time a new ppp connection is
# established. This might be useful, if you are using dial-on-demand.
run_ipup="false"
# Set to "true" if ddclient should run in daemon mode
# If this is changed to true, run_ipup and run_dhclient must be set to false.
run_daemon="true"
# Set the time interval between the updates of the dynamic DNS name in seconds.
# This option only takes effect if the ddclient runs in daemon mode.
daemon_interval="300"
```
5. Restart the `ddclient` service - twice for good measure (daemon mode only gets activated on the second go *because reasons*):
```shell
$ sudo systemctl restart ddclient
$ sudo systemctl restart ddclient
```
6. After a few moments, refresh the Google Domains page to verify that your instance's external IP address is showing up on the new DDNS record.
### Install Docker
*Steps taken from [here](https://docs.docker.com/install/linux/docker-ce/debian/).*
1. Update `apt` package index:
```shell
$ sudo apt-get update
```
2. Install package management prereqs:
```shell
$ sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common
```
3. Add Docker GPG key:
```shell
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
```
4. Add the Docker repo:
```shell
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
```
5. Update apt index again:
```shell
$ sudo apt-get update
```
6. Install Docker:
```shell
$ sudo apt-get install docker-ce
```
### Install Certbot and generate SSL cert
*Steps taken from [here](https://certbot.eff.org/instructions?ws=other&os=debianbuster).*
1. Install Certbot:
```shell
$ sudo apt-get install certbot
```
2. Generate certificate:
```shell
$ sudo certbot certonly --standalone -d [FQDN]
```
3. Create a directory to store the new certificates and copy them there:
```shell
$ sudo mkdir -p /ssl/keys/
$ sudo cp -p /etc/letsencrypt/live/[FQDN]/fullchain.pem /ssl/keys/
$ sudo cp -p /etc/letsencrypt/live/[FQDN]/privkey.pem /ssl/keys/
```
### Set up vaultwarden
*Using the container image available [here](https://github.com/dani-garcia/vaultwarden).*
1. Let's just get it up and running first:
```shell
$ sudo docker run -d --name vaultwarden \
-e ROCKET_TLS={certs='"/ssl/fullchain.pem", key="/ssl/privkey.pem"}' \
-e ROCKET_PORT='8000' \
-v /ssl/keys/:/ssl/ \
-v /bw-data/:/data/ \
-v /icon_cache/ \
-p 0.0.0.0:443:8000 \
vaultwarden/server:latest
```
2. At this point you should be able to point your web browser at `https://[FQDN]` and see the BitWarden login screen. Click on the Create button and set up a new account. Log in, look around, add some passwords, etc. Everything should basically work just fine.
3. Unless you want to host passwords for all of the Internet you'll probably want to disable signups at some point by adding the `env` option `SIGNUPS_ALLOWED=false`. And you'll need to set `DOMAIN=https://[FQDN]` if you want to use U2F authentication:
```shell
$ sudo docker stop vaultwarden
$ sudo docker rm vaultwarden
$ sudo docker run -d --name vaultwarden \
-e ROCKET_TLS={certs='"/ssl/fullchain.pem",key="/ssl/privkey.pem"'} \
-e ROCKET_PORT='8000' \
-e SIGNUPS_ALLOWED=false \
-e DOMAIN=https://[FQDN] \
-v /ssl/keys/:/ssl/ \
-v /bw-data/:/data/ \
-v /icon_cache/ \
-p 0.0.0.0:443:8000 \
vaultwarden/server:latest
```
### Install vaultwarden as a service
*So we don't have to keep manually firing this thing off.*
1. Create a script to stop, remove, update, and (re)start the `vaultwarden` container:
```shell
$ sudo vi /usr/local/bin/start-vaultwarden.sh
#!/bin/bash
docker stop vaultwarden
docker rm vaultwarden
docker pull vaultwarden/server
docker run -d --name vaultwarden \
-e ROCKET_TLS={certs='"/ssl/fullchain.pem",key="/ssl/privkey.pem"'} \
-e ROCKET_PORT='8000' \
-e SIGNUPS_ALLOWED=false \
-e DOMAIN=https://[FQDN] \
-v /ssl/keys/:/ssl/ \
-v /bw-data/:/data/ \
-v /icon_cache/ \
-p 0.0.0.0:443:8000 \
vaultwarden/server:latest
$ sudo chmod 744 /usr/local/bin/start-vaultwarden.sh
```
2. And add it as a `systemd` service:
```shell
$ sudo vi /etc/systemd/system/vaultwarden.service
[Unit]
Description=BitWarden container
Requires=docker.service
After=docker.service
[Service]
Restart=always
ExecStart=/usr/local/bin/vaultwarden-start.sh
ExecStop=/usr/bin/docker stop vaultwarden
[Install]
WantedBy=default.target
$ sudo chmod 644 /etc/systemd/system/vaultwarden.service
```
3. Try it out:
```shell
$ sudo systemctl start vaultwarden
$ sudo systemctl status vaultwarden
● bitwarden.service - BitWarden container
Loaded: loaded (/etc/systemd/system/vaultwarden.service; enabled; vendor preset: enabled)
Active: deactivating (stop) since Sun 2018-09-09 03:43:20 UTC; 1s ago
Process: 13104 ExecStart=/usr/local/bin/bitwarden-start.sh (code=exited, status=0/SUCCESS)
Main PID: 13104 (code=exited, status=0/SUCCESS); Control PID: 13229 (docker)
Tasks: 5 (limit: 4915)
Memory: 9.7M
CPU: 375ms
CGroup: /system.slice/vaultwarden.service
└─control
└─13229 /usr/bin/docker stop vaultwarden
Sep 09 03:43:20 vaultwarden vaultwarden-start.sh[13104]: Status: Image is up to date for vaultwarden/server:latest
Sep 09 03:43:20 vaultwarden vaultwarden-start.sh[13104]: ace64ca5294eee7e21be764ea1af9e328e944658b4335ce8721b99a33061d645
```
### Conclusion
If all went according to plan, you've now got a highly-secure open-source full-featured cross-platform password manager running on an Always Free Google Compute Engine instance resolved by Google Domains dynamic DNS. Very slick!

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

View file

@ -0,0 +1,33 @@
---
series: Tips
date: "2020-12-23T08:34:30Z"
thumbnail: -lp1-DGiM.png
tags:
- chromeos
title: Burn an ISO to USB with the Chromebook Recovery Utility
toc: false
featured: true
---
There are a number of fantastic Windows applications for creating bootable USB drives from ISO images - but those don't work on a Chromebook. Fortunately there's an easily-available tool which will do the trick: Google's own [Chromebook Recovery Utility](https://chrome.google.com/webstore/detail/chromebook-recovery-utili/pocpnlppkickgojjlmhdmidojbmbodfm) app.
Normally that tool is used to creating bootable media to [reinstall Chrome OS on a broken Chromebook](https://support.google.com/chromebook/answer/1080595) (hence the name) but it also has the capability to write other arbitrary images as well. So if you find yourself needing to create a USB drive for installing ESXi on a computer in your [home lab](https://twitter.com/johndotbowdre/status/1341767090945077248) (more on that soon!) here's what you'll need to do:
1. Install the [Chromebook Recovery Utility](https://chrome.google.com/webstore/detail/chromebook-recovery-utili/pocpnlppkickgojjlmhdmidojbmbodfm).
2. Download the ISO you intend to use.
3. Rename the file to append `.bin` on the end, after the `.iso` bit:
![My renamed ISO for installing ESXi](uoTjgtbN1.png)
4. Plug in the USB drive you're going to sacrifice for this effort - remember that ALL data on the drive will be erased.
5. Open the recovery utility, click on the gear icon at the top right, and select the *Use local image* option:
![The CRU menu](vdTpW9t7Q.png)
6. Browse to and select the `*.iso.bin` file.
7. Choose the USB drive, and click *Continue*.
![Selecting the drive](p_Ieqsw4p.png)
8. Click *Create now* to start the writing!
![Writing the image](lhw5EEqSD.png)
9. All done! It probably won't work great for actually recovering your Chromebook but will do wonders for installing ESXi (or whatever) on another computer!
![Success!](-lp1-DGiM.png)
You can also use the CRU to make a bootable USB from a `.zip` archive containing a single `.img` file, such as those commonly used to distribute [Raspberry Pi images](https://www.raspberrypi.org/documentation/installation/installing-images/chromeos.md).
Very cool!

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

View file

@ -0,0 +1,54 @@
---
title: "Cat a File Without Comments" # Title of the blog post.
date: 2023-02-22 # Date of post creation.
# lastmod: 2023-02-20T10:32:20-06:00 # Date when last modified
description: "A quick trick to strip out the comments when viewing the contents of a file." # Description used for search engine.
featured: false # Sets if post is a featured post, making appear on the home page side bar.
draft: false # Sets whether to render this page. Draft of true will not be rendered.
toc: true # Controls if a table of contents should be generated for first-level links automatically.
usePageBundles: true
# menu: main
# featureImage: "file.png" # Sets featured image on blog post.
# featureImageAlt: 'Description of image' # Alternative text for featured image.
# featureImageCap: 'This is the featured image.' # Caption (optional).
# thumbnail: "thumbnail.png" # Sets thumbnail image appearing inside card on homepage.
# shareImage: "share.png" # Designate a separate image for social media sharing.
codeLineNumbers: false # Override global value for showing of line numbers within code block.
series: Tips # Projects, Scripts, vRA8, K8s on vSphere
tags:
- linux
- shell
- regex
comment: true # Disable comment if false.
---
It's super handy when a Linux config file is loaded with comments to tell you precisely how to configure the thing, but all those comments can really get in the way when you're trying to review the current configuration.
Next time, instead of scrolling through page after page of lengthy embedded explanations, just use:
```shell
egrep -v "^\s*(#|$)" $filename
```
For added usefulness, I alias this command to `ccat` (which my brain interprets as "commentless cat") in [my `~/.zshrc`](https://github.com/jbowdre/dotfiles/blob/main/zsh/.zshrc):
```shell
alias ccat='egrep -v "^\s*(#|$)"'
```
Now instead of viewing all 75 lines of a [mostly-default Vagrantfile](/create-vms-chromebook-hashicorp-vagrant), I just see the 7 that matter:
```shell
; wc -l Vagrantfile
75 Vagrantfile
; ccat Vagrantfile
Vagrant.configure("2") do |config|
config.vm.box = "oopsme/windows11-22h2"
config.vm.provider :libvirt do |libvirt|
libvirt.cpus = 4
libvirt.memory = 4096
end
end
; ccat Vagrantfile | wc -l
7
```
Nice!

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 188 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 555 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

View file

@ -0,0 +1,503 @@
---
series: Projects
date: "2021-10-28T00:00:00Z"
thumbnail: 20211028_wireguard_in_the_cloud.jpg
usePageBundles: true
tags:
- linux
- gcp
- cloud
- wireguard
- vpn
- homelab
- tasker
- automation
- networking
- security
title: Cloud-hosted WireGuard VPN for remote homelab access
featured: false
---
For a while now, I've been using an [OpenVPN Access Server](https://openvpn.net/access-server/) virtual appliance for remotely accessing my [homelab](/vmware-home-lab-on-intel-nuc-9). That's worked _fine_ but it comes with a lot of overhead. It also requires maintaining an SSL certificate and forwarding three ports through my home router, in addition to managing a fairly complex software package and configurations. The free version of the OpenVPN server also only supports a maximum of two simultaneous connections. I recently ran into issues with the `certbot` automated SSL renewal process on my OpenVPN AS VM and decided that it might be time to look for a simpler solution.
I found that solution in [WireGuard](https://www.wireguard.com/), which provides an extremely efficient secure tunnel implemented directly in the Linux kernel. It has a much smaller (and easier-to-audit) codebase, requires minimal configuration, and uses the latest crypto wizardry to securely connect multiple systems. It took me an hour or so of fumbling to get WireGuard deployed and configured on a fresh (and minimal) Ubuntu 20.04 VM running on my ESXi 7 homelab host, and I was pretty happy with the performance, stability, and resource usage of the new setup. That new VM idled at a full _tenth_ of the memory usage of my OpenVPN AS, and it only required a single port to be forwarded into my home network.
Of course, I soon realized that the setup could be _even better:_ I'm now running a WireGuard server on the Google Cloud free tier, and I've configured the [VyOS virtual router I use for my homelab stuff](/vmware-home-lab-on-intel-nuc-9#networking) to connect to that cloud-hosted server to create a secure tunnel between the two without needing to punch any holes in my local network (or consume any additional resources). I can then connect my client devices to the WireGuard server in the cloud. From there, traffic intended for my home network gets relayed to the VyOS router, and internet-bound traffic leaves Google Cloud directly. So my self-managed VPN isn't just good for accessing my home lab remotely, but also more generally for encrypting traffic when on WiFi networks I don't control - allowing me to replace the paid ProtonVPN subscription I had been using for that purpose.
It's a pretty slick setup, if I do say so myself. Anyway, this post will discuss how I implemented this, and what I learned along the way.
### WireGuard Concepts, in Brief
WireGuard does things a bit differently from other VPN solutions I've used in the past. For starters, there aren't any user accounts to manage, and in fact users don't really come into the picture at all. WireGuard also doesn't really distinguish between _client_ and _server_; the devices on both ends of a tunnel connection are _peers_, and they use the same software package and very similar configurations. Each WireGuard peer is configured with a virtual network interface with a private IP address used for the tunnel network, and a configuration file tells it which tunnel IP(s) will be used by the other peer(s). Each peer has its own cryptographic _private_ key, and the other peers get a copy of the corresponding _public_ key added to their configuration so that all the peers can recognize each other and encrypt/decrypt traffic appropriately. This mapping of peer addresses to public keys facilitates what WireGuard calls [Cryptokey Routing](https://www.wireguard.com/#cryptokey-routing).
Once the peers are configured, all it takes is bringing up the WireGuard virtual interface on each peer to establish the tunnel and start passing secure traffic.
You can read a lot more fascinating details about how this all works back on the [WireGuard homepage](https://www.wireguard.com/#conceptual-overview) (and even more in this [protocol description](https://www.wireguard.com/protocol/)) but this at least covers the key points I needed to grok prior to a successful initial deployment.
For my hybrid cloud solution, I also leaned heavily upon [this write-up of a WireGuard Site-to-Site configuration](https://gist.github.com/insdavm/b1034635ab23b8839bf957aa406b5e39) for how to get traffic flowing between my on-site environment, cloud-hosted WireGuard server, and "Road Warrior" client devices, and drew from [this documentation on implementing WireGuard in GCP](https://github.com/agavrel/wireguard_google_cloud) as well. The [VyOS documentation for configuring the built-in WireGuard interface](https://docs.vyos.io/en/latest/configuration/interfaces/wireguard.html) was also quite helpful to me.
Okay, enough background; let's get this thing going.
### Google Cloud Setup
#### Instance Deployment
I started by logging into my Google Cloud account at https://console.cloud.google.com, and proceeded to create a new project (named `wireguard`) to keep my WireGuard-related resources together. I then navigated to **Compute Engine** and [created a new instance](https://console.cloud.google.com/compute/instancesAdd) inside that project. The basic setup is:
| Attribute | Value |
| --- | --- |
| Name | `wireguard` |
| Region | `us-east1` (or whichever [free-tier-eligible region](https://cloud.google.com/free/docs/gcp-free-tier/#compute) is closest) |
| Machine Type | `e2-micro` |
| Boot Disk Size | 10 GB |
| Boot Disk Image | Ubuntu 20.04 LTS |
![Instance creation](20211027_instance_creation.png)
The other defaults are fine, but I'll holding off on clicking the friendly blue "Create" button at the bottom and instead click to expand the **Networking, Disks, Security, Management, Sole-Tenancy** sections to tweak a few more things.
![Instance creation advanced settings](20211028_instance_advanced_settings.png)
##### Network Configuration
Expanding the **Networking** section of the request form lets me add a new `wireguard` network tag, which will make it easier to target the instance with a firewall rule later. I also want to enable the _IP Forwarding_ option so that the instance will be able to do router-like things.
By default, the new instance will get assigned a public IP address that I can use to access it externally - but this address is _ephemeral_ so it will change periodically. Normally I'd overcome this by [using ddclient to manage its dynamic DNS record](/bitwarden-password-manager-self-hosted-on-free-google-cloud-instance#configure-dynamic-dns), but (looking ahead) [VyOS's WireGuard interface configuration](https://docs.vyos.io/en/latest/configuration/interfaces/wireguard.html#interface-configuration) unfortunately only supports connecting to an IP rather than a hostname. That means I'll need to reserve a _static_ IP address for my instance.
I can do that by clicking on the _Default_ network interface to expand the configuration. While I'm here, I'll first change the **Network Service Tier** from _Premium_ to _Standard_ to save a bit of money on network egress fees. _(This might be a good time to mention that while the compute instance itself is free, I will have to spend [about $3/mo for the public IP](https://cloud.google.com/vpc/network-pricing#:~:text=internal%20IP%20addresses.-,External%20IP%20address%20pricing,-You%20are%20charged), as well as [$0.085/GiB for internet egress via the Standard tier](https://cloud.google.com/vpc/network-pricing#:~:text=or%20Cloud%20Interconnect.-,Standard%20Tier%20pricing,-Egress%20pricing%20is) (versus [$0.12/GiB on the Premium tier](https://cloud.google.com/vpc/network-pricing#:~:text=Premium%20Tier%20pricing)). So not entirely free, but still pretty damn cheap for a cloud-hosted VPN that I control completely.)_
Anyway, after switching to the cheaper Standard tier I can click on the **External IP** dropdown and select the option to _Create IP Address_. I give it the same name as my instance to make it easy to keep up with.
![Network configuration](20211027_network_settings.png)
##### Security Configuration
The **Security** section lets me go ahead and upload an SSH public key that I can then use for logging into the instance once it's running. Of course, that means I'll first need to generate a key pair for this purpose:
```sh
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_wireguard
```
Okay, now that I've got my keys, I can click the **Add Item** button and paste in the contents of `~/.ssh/id_ed25519_wireguard.pub`.
![Security configuration](20211027_security_settings.png)
And that's it for the pre-deploy configuration! Time to hit **Create** to kick it off.
![Do it!](20211027_creation_time.png)
The instance creation will take a couple of minutes but I can go ahead and get the firewall sorted while I wait.
#### Firewall
Google Cloud's default firewall configuration will let me reach my new server via SSH without needing to configure anything, but I'll need to add a new rule to allow the WireGuard traffic. I do this by going to **VPC > Firewall** and clicking the button at the top to **[Create Firewall Rule](https://console.cloud.google.com/networking/firewalls/add)**. I give it a name (`allow-wireguard-ingress`), select the rule target by specifying the `wireguard` network tag I had added to the instance, and set the source range to `0.0.0.0/0`. I'm going to use the default WireGuard port so select the _udp:_ checkbox and enter `51820`.
![Firewall rule creation](20211027_firewall.png)
I'll click **Create** and move on.
#### WireGuard Server Setup
Once the **Compute Engine > Instances** [page](https://console.cloud.google.com/compute/instances) indicates that the instance is ready, I can make a note of the listed public IP and then log in via SSH:
```sh
ssh -i ~/.ssh/id_25519_wireguard {PUBLIC_IP}
```
##### Preparation
And, as always, I'll first make sure the OS is fully updated before doing anything else:
```sh
sudo apt update
sudo apt upgrade
```
Then I'll install `ufw` to easily manage the host firewall, `qrencode` to make it easier to generate configs for mobile clients, `openresolv` to avoid [this issue](https://superuser.com/questions/1500691/usr-bin-wg-quick-line-31-resolvconf-command-not-found-wireguard-debian/1500896), and `wireguard` to, um, guard the wires:
```sh
sudo apt install ufw qrencode openresolv wireguard
```
Configuring the host firewall with `ufw` is very straight forward:
```sh
# First, SSH:
sudo ufw allow 22/tcp
# and WireGuard:
sudo ufw allow 51820/udp
# Then turn it on:
sudo ufw enable
```
The last preparatory step is to enable packet forwarding in the kernel so that the instance will be able to route traffic between the remote clients and my home network (once I get to that point). I can configure that on-the-fly with:
```sh
sudo sysctl -w net.ipv4.ip_forward=1
```
To make it permanent, I'll edit `/etc/sysctl.conf` and uncomment the same line:
```sh
$ sudo vi /etc/sysctl.conf
# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1
```
##### WireGuard Interface Config
I'll switch to the root user, move into the `/etc/wireguard` directory, and issue `umask 077` so that the files I'm about to create will have a very limited permission set (to be accessible by root, and _only_ root):
```sh
sudo -i
cd /etc/wireguard
umask 077
```
Then I can use the `wg genkey` command to generate the server's private key, save it to a file called `server.key`, pass it through `wg pubkey` to generate the corresponding public key, and save that to `server.pub`:
```sh
wg genkey | tee server.key | wg pubkey > server.pub
```
As I mentioned earlier, WireGuard will create a virtual network interface using an internal network to pass traffic between the WireGuard peers. By convention, that interface is `wg0` and it draws its configuration from a file in `/etc/wireguard` named `wg0.conf`. I could create a configuration file with a different name and thus wind up with a different interface name as well, but I'll stick with tradition to keep things easy to follow.
The format of the interface configuration file will need to look something like this:
```
[Interface] # this section defines the local WireGuard interface
Address = # CIDR-format IP address of the virtual WireGuard interface
ListenPort = # WireGuard listens on this port for incoming traffic (randomized if not specified)
PrivateKey = # private key used to encrypt traffic sent to other peers
MTU = # packet size
DNS = # optional DNS server(s) and search domain(s) used for the VPN
PostUp = # command executed by wg-quick wrapper when the interface comes up
PostDown = # command executed by wg-quick wrapper when the interface goes down
[Peer] # now we're talking about the other peers connecting to this instance
PublicKey = # public key used to decrypt traffic sent by this peer
AllowedIPs = # which IPs will be routed to this peer
```
There will be a single `[Interface]` section in each peer's configuration file, but they may include multiple `[Peer]` sections. For my config, I'll use the `10.200.200.0/24` network for WireGuard, and let this server be `10.200.200.1`, the VyOS router in my home lab `10.200.200.2`, and I'll assign IPs to the other peers from there. I found a note that Google Cloud uses an MTU size of `1460` bytes so that's what I'll set on this end. I'm going to configure WireGuard to use the VyOS router as the DNS server, and I'll specify my internal `lab.bowdre.net` search domain. Finally, I'll leverage the `PostUp` and `PostDown` directives to enable and disable NAT so that the server will be able to forward traffic between networks for me.
So here's the start of my GCP WireGuard server's `/etc/wireguard/wg0.conf`:
```sh
# /etc/wireguard/wg0.conf
[Interface]
Address = 10.200.200.1/24
ListenPort = 51820
PrivateKey = {GCP_PRIVATE_KEY}
MTU = 1460
DNS = 10.200.200.2, lab.bowdre.net
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o ens4 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o ens4 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o ens4 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o ens4 -j MASQUERADE
```
I don't have any other peers ready to add to this config yet, but I can go ahead and bring up the interface all the same. I'm going to use the `wg-quick` wrapper instead of calling `wg` directly since it simplifies a bit of the configuration, but first I'll need to enable the `wg-quick@{INTERFACE}` service so that it will run automatically at startup:
```sh
systemctl enable wg-quick@wg0
systemctl start wg-quick@wg0
```
I can now bring up the interface with `wg-quick up wg0` and check the status with `wg show`:
```
root@wireguard:~# wg-quick up wg0
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.200.200.1/24 dev wg0
[#] ip link set mtu 1460 up dev wg0
[#] resolvconf -a wg0 -m 0 -x
[#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o ens4 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o ens4 -j MASQUERADE
root@wireguard:~# wg show
interface: wg0
public key: {GCP_PUBLIC_IP}
private key: (hidden)
listening port: 51820
```
I'll come back here once I've got a peer config to add.
### Configure VyoS Router as WireGuard Peer
Comparatively, configuring WireGuard on VyOS is a bit more direct. I'll start by entering configuration mode and generating and binding a key pair for this interface:
```sh
configure
run generate pki wireguard key-pair install interface wg0
```
And then I'll configure the rest of the options needed for the interface:
```sh
set interfaces wireguard wg0 address '10.200.200.2/24'
set interfaces wireguard wg0 description 'VPN to GCP'
set interfaces wireguard wg0 peer wireguard-gcp address '{GCP_PUBLIC_IP}'
set interfaces wireguard wg0 peer wireguard-gcp allowed-ips '0.0.0.0/0'
set interfaces wireguard wg0 peer wireguard-gcp persistent-keepalive '25'
set interfaces wireguard wg0 peer wireguard-gcp port '51820'
set interfaces wireguard wg0 peer wireguard-gcp public-key '{GCP_PUBLIC_KEY}'
```
Note that this time I'm allowing all IPs (`0.0.0.0/0`) so that this WireGuard interface will pass traffic intended for any destination (whether it's local, remote, or on the Internet). And I'm specifying a [25-second `persistent-keepalive` interval](https://www.wireguard.com/quickstart/#nat-and-firewall-traversal-persistence) to help ensure that this NAT-ed tunnel stays up even when it's not actively passing traffic - after all, I'll need the GCP-hosted peer to be able to initiate the connection so I can access the home network remotely.
While I'm at it, I'll also add a static route to ensure traffic for the WireGuard tunnel finds the right interface:
```sh
set protocols static route 10.200.200.0/24 interface wg0
```
And I'll add the new `wg0` interface as a listening address for the VyOS DNS forwarder:
```sh
set service dns forwarding listen-address '10.200.200.2'
```
I can use the `compare` command to verify the changes I've made, and then apply and save the updated config:
```sh
compare
commit
save
```
I can check the status of WireGuard on VyOS (and view the public key!) like so:
```sh
$ show interfaces wireguard wg0 summary
interface: wg0
public key: {VYOS_PUBLIC_KEY}
private key: (hidden)
listening port: 43543
peer: {GCP_PUBLIC_KEY}
endpoint: {GCP_PUBLIC_IP}:51820
allowed ips: 0.0.0.0/0
transfer: 0 B received, 592 B sent
persistent keepalive: every 25 seconds
```
See? That part was much easier to set up! But it doesn't look like it's actually passing traffic yet... because while the VyOS peer has been configured with the GCP peer's public key, the GCP peer doesn't know anything about the VyOS peer yet.
So I'll copy `{VYOS_PUBLIC_KEY}` and SSH back to the GCP instance to finish that configuration. Once I'm there, I can edit `/etc/wireguard/wg0.conf` as root and add in a new `[Peer]` section at the bottom, like this:
```
[Peer]
# VyOS
PublicKey = {VYOS_PUBLIC_KEY}
AllowedIPs = 10.200.200.2/32, 192.168.1.0/24, 172.16.0.0/16
```
This time, I'm telling WireGuard that the new peer has IP `10.200.200.2` but that it should also get traffic destined for the `192.168.1.0/24` and `172.16.0.0/16` networks, my home and lab networks. Again, the `AllowedIPs` parameter is used for WireGuard's Cryptokey Routing so that it can keep track of which traffic goes to which peers (and which key to use for encryption).
After saving the file, I can either restart WireGuard by bringing the interface down and back up (`wg-quick down wg0 && wg-quick up wg0`), or I can reload it on the fly with:
```sh
sudo -i
wg syncconf wg0 <(wg-quick strip wg0)
```
(I can't just use `wg syncconf wg0` directly since `/etc/wireguard/wg0.conf` includes the `PostUp`/`PostDown` commands which can only be parsed by the `wg-quick` wrapper, so I'm using `wg-quick strip {INTERFACE}` to grab the contents of the config file, remove the problematic bits, and then pass what's left to the `wg syncconf {INTERFACE}` command to update the current running config.)
Now I can check the status of WireGuard on the GCP end:
```sh
root@wireguard:~# wg show
interface: wg0
public key: {GCP_PUBLIC_KEY}
private key: (hidden)
listening port: 51820
peer: {VYOS_PUBLIC_KEY}
endpoint: {VYOS_PUBLIC_IP}:43990
allowed ips: 10.200.200.2/32, 192.168.1.0/24, 172.16.0.0/16
latest handshake: 55 seconds ago
transfer: 1.23 KiB received, 368 B sent
```
Hey, we're passing traffic now! And I can verify that I can ping stuff on my home and lab networks from the GCP instance:
```sh
john@wireguard:~$ ping -c 1 192.168.1.5
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=127 time=35.6 ms
--- 192.168.1.5 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 35.598/35.598/35.598/0.000 ms
john@wireguard:~$ ping -c 1 172.16.10.1
PING 172.16.10.1 (172.16.10.1) 56(84) bytes of data.
64 bytes from 172.16.10.1: icmp_seq=1 ttl=64 time=35.3 ms
--- 172.16.10.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 35.275/35.275/35.275/0.000 ms
```
Cool!
### Adding Additional Peers
So my GCP and VyOS peers are talking, but the ultimate goals here are for my Chromebook to have access to my homelab resources while away from home, and for my phones to have secure internet access when connected to WiFi networks I don't control. That means adding at least two more peers to the GCP server. WireGuard [offers downloads](https://www.wireguard.com/install/) for just about every operating system you can imagine, but I'll be using the [Android app](https://play.google.com/store/apps/details?id=com.wireguard.android) for both the Chromebook and phones.
#### Chromebook
The first step is to install the WireGuard Android app.
_Note: the version of the WireGuard app currently available on the Play Store (v1.0.20210926) [has an issue](https://www.reddit.com/r/WireGuard/comments/q11rt9/wireguard_1020210926_and_chromeos/) on Chrome OS that causes it to not pass traffic after the Chromebook has resumed from sleep. The workaround for this is to install an older version of the app (1.0.20210506) which can be obtained from [F-Droid](https://f-droid.org/en/packages/com.wireguard.android/). Doing so requires having the Linux environment enabled on Chrome OS and the **Develop Android Apps > Enable ADB Debugging** option enabled in the Chrome OS settings. The process for sideloading apps is [detailed here](https://developer.android.com/topic/arc/development-environment)._
Once it's installed, I open the app and click the "Plus" button to create a new tunnel, and select the _Create from scratch_ option. I click the circle-arrows icon at the right edge of the _Private key_ field, and that automatically generates this peer's private and public key pair. Simply clicking on the _Public key_ field will automatically copy the generated key to my clipboard, which will be useful for sharing it with the server. Otherwise I fill out the **Interface** section similarly to what I've done already:
| Parameter | Value |
| --- | --- |
| Name | `wireguard-gcp` |
| Private key | `{CB_PRIVATE_KEY}` |
| Public key | `{CB_PUBLIC_KEY}` |
| Addresses | `10.200.200.3/24` |
| Listen port | |
| DNS servers | `10.200.200.2` |
| MTU | |
I then click the **Add Peer** button to tell this client about the peer it will be connecting to - the GCP-hosted instance:
| Parameter | Value |
| --- | --- |
| Public key | `{GCP_PUBLIC_KEY}` |
| Pre-shared key | |
| Persistent keepalive | |
| Endpoint | `{GCP_PUBLIC_IP}:51820` |
| Allowed IPs | `0.0.0.0/0` |
I _shouldn't_ need the keepalive for the "Road Warrior" peers connecting to the GCP peer, but I can always set that later if I run into stability issues.
Now I can go ahead and save this configuration, but before I try (and fail) to connect I first need to tell the cloud-hosted peer about the Chromebook. So I fire up an SSH session to my GCP instance, become root, and edit the WireGuard configuration to add a new `[Peer]` section.
```sh
sudo -i
vi /etc/wireguard/wg0.conf
```
Here's the new section that I'll add to the bottom of the config:
```sh
[Peer]
# Chromebook
PublicKey = {CB_PUBLIC_KEY}
AllowedIPs = 10.200.200.3/32
```
This one is acting as a single-node endpoint (rather than an entryway into other networks like the VyOS peer) so setting `AllowedIPs` to only the peer's IP makes sure that WireGuard will only send it traffic specifically intended for this peer.
So my complete `/etc/wireguard/wg0.conf` looks like this so far:
```sh
# /etc/wireguard/wg0.conf
[Interface]
Address = 10.200.200.1/24
ListenPort = 51820
PrivateKey = {GCP_PRIVATE_KEY}
MTU = 1460
DNS = 10.200.200.2, lab.bowdre.net
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o ens4 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o ens4 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o ens4 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o ens4 -j MASQUERADE
[Peer]
# VyOS
PublicKey = {VYOS_PUBLIC_KEY}
AllowedIPs = 10.200.200.2/32, 192.168.1.0/24, 172.16.0.0/16
[Peer]
# Chromebook
PublicKey = {CB_PUBLIC_KEY}
AllowedIPs = 10.200.200.3/32
```
Now to save the file and reload the WireGuard configuration again:
```sh
wg syncconf wg0 <(wg-quick strip wg0)
```
At this point I can activate the connection in the WireGuard Android app, wait a few seconds, and check with `wg show` to confirm that the tunnel has been established successfully:
```sh
root@wireguard:~# wg show
interface: wg0
public key: {GCP_PUBLIC_KEY}
private key: (hidden)
listening port: 51820
peer: {VYOS_PUBLIC_KEY}
endpoint: {VYOS_PUBLIC_IP}:43990
allowed ips: 10.200.200.2/32, 192.168.1.0/24, 172.16.0.0/16
latest handshake: 1 minute, 55 seconds ago
transfer: 200.37 MiB received, 16.32 MiB sent
peer: {CB_PUBLIC_KEY}
endpoint: {CB_PUBLIC_IP}:33752
allowed ips: 10.200.200.3/32
latest handshake: 48 seconds ago
transfer: 169.17 KiB received, 808.33 KiB sent
```
And I can even access my homelab when not at home!
![Remote access to my homelab!](20211028_remote_homelab.png)
#### Android Phone
Being able to copy-and-paste the required public keys between the WireGuard app and the SSH session to the GCP instance made it relatively easy to set up the Chromebook, but things could be a bit trickier on a phone without that kind of access. So instead I will create the phone's configuration on the WireGuard server in the cloud, render that config file as a QR code, and simply scan that through the phone's WireGuard app to import the settings.
I'll start by SSHing to the GCP instance, elevating to root, setting the restrictive `umask` again, and creating a new folder to store client configurations.
```sh
sudo -i
umask 077
mkdir /etc/wireguard/clients
cd /etc/wireguard/clients
```
As before, I'll use the built-in `wg` commands to generate the private and public key pair:
```sh
wg genkey | tee phone1.key | wg pubkey > phone1.pub
```
I can then use those keys to assemble the config for the phone:
```sh
# /etc/wireguard/clients/phone1.conf
[Interface]
PrivateKey = {PHONE1_PRIVATE_KEY}
Address = 10.200.200.4/24
DNS = 10.200.200.2, lab.bowdre.net
[Peer]
PublicKey = {GCP_PUBLIC_KEY}
AllowedIPs = 0.0.0.0/0
Endpoint = {GCP_PUBLIC_IP}:51820
```
I'll also add the interface address and corresponding public key to a new `[Peer]` section of `/etc/wireguard/wg0.conf`:
```sh
[Peer]
PublicKey = {PHONE1_PUBLIC_KEY}
AllowedIPs = 10.200.200.4/32
```
And reload the WireGuard config:
```sh
wg syncconf wg0 <(wg-quick strip wg0)
```
Back in the `clients/` directory, I can use `qrencode` to render the phone configuration file (keys and all!) as a QR code:
```sh
qrencode -t ansiutf8 < phone1.conf
```
![QR code config](20211028_qrcode_config.png)
And then I just open the WireGuard app on my phone and use the **Scan from QR Code** option. After a successful scan, it'll prompt me to name the new tunnel, and then I should be able to connect right away.
![Successful mobile connection](20211028_wireguard_mobile.png)
I can even access my vSphere lab environment - not that it offers a great mobile experience...
![vSphere mobile sucks](20211028_mobile_vsphere_sucks.jpg)
Before moving on too much further, though, I'm going to clean up the keys and client config file that I generated on the GCP instance. It's not great hygiene to keep a private key stored on the same system it's used to access.
```sh
rm -f /etc/wireguard/clients/*
```
##### Bonus: Automation!
I've [written before](auto-connect-to-protonvpn-on-untrusted-wifi-with-tasker) about a set of [Tasker](https://play.google.com/store/apps/details?id=net.dinglisch.android.taskerm) profiles I put together so that my phone would automatically connect to a VPN whenever it connects to a WiFi network I don't control. It didn't take much effort at all to adapt the profile to work with my new WireGuard setup.
Two quick pre-requisites first:
1. Open the WireGuard Android app, tap the three-dot menu button at the top right, expand the Advanced section, and enable the _Allow remote control apps_ so that Tasker will be permitted to control WireGuard.
2. Exclude the WireGuard app from Android's battery optimization so that it doesn't have any problems running in the background. On (Pixel-flavored) Android 12, this can be done by going to **Settings > Apps > See all apps > WireGuard > Battery** and selecting the _Unrestricted_ option.
On to the Tasker config. The only changes will be in the [VPN on Strange Wifi](/auto-connect-to-protonvpn-on-untrusted-wifi-with-tasker#vpn-on-strange-wifi) profile. I'll remove the OpenVPN-related actions from the Enter and Exit tasks and replace them with the built-in **Tasker > Tasker Function WireGuard Set Tunnel** action.
For the Enter task, I'll set the tunnel status to `true` and specify the name of the tunnel as configured in the WireGuard app; the Exit task gets the status set to `false` to disable the tunnel. Both actions will be conditional upon the `%TRUSTED_WIFI` variable being unset.
![Tasker setup](20211028_tasker_setup.png)
```
Profile: VPN on Strange WiFi
Settings: Notification: no
State: Wifi Connected [ SSID:* MAC:* IP:* Active:Any ]
Enter Task: ConnectVPN
A1: Tasker Function [
Function: WireGuardSetTunnel(true,wireguard-gcp) ]
If [ %TRUSTED_WIFI !Set ]
Exit Task: DisconnectVPN
A1: Tasker Function [
Function: WireGuardSetTunnel(false,wireguard-gcp) ]
If [ %TRUSTED_WIFI !Set ]
```
_Automagic!_
#### Other Peers
Any additional peers that need to be added in the future will likely follow one of the above processes. The steps are always to generate the peer's key pair, use the private key to populate the `[Interface]` portion of the peer's config, configure the `[Peer]` section with the _public_ key, allowed IPs, and endpoint address of the peer it will be connecting to, and then to add the new peer's _public_ key and internal WireGuard IP to a new `[Peer]` section of the existing peer's config.

View file

@ -0,0 +1,248 @@
---
title: "Create Virtual Machines on a Chromebook with HashiCorp Vagrant" # Title of the blog post.
date: 2023-02-20 # Date of post creation.
lastmod: 2023-02-25
description: "Pairing the powerful Linux Development Environment on modern Chromebooks with HashiCorp Vagrant to create and manage local virtual machines for development and testing" # Description used for search engine.
featured: true # Sets if post is a featured post, making appear on the home page side bar.
draft: false # Sets whether to render this page. Draft of true will not be rendered.
toc: true # Controls if a table of contents should be generated for first-level links automatically.
usePageBundles: true
# menu: main
# featureImage: "file.png" # Sets featured image on blog post.
# featureImageAlt: 'Description of image' # Alternative text for featured image.
# featureImageCap: 'This is the featured image.' # Caption (optional).
thumbnail: "thumbnail.png" # Sets thumbnail image appearing inside card on homepage.
# shareImage: "share.png" # Designate a separate image for social media sharing.
codeLineNumbers: false # Override global value for showing of line numbers within code block.
series: Projects
tags:
- linux
- chromeos
- homelab
- infrastructure-as-code
comment: true # Disable comment if false.
---
I've lately been trying to do more with [Salt](https://saltproject.io/) at work, but I'm still very much a novice with that tool. I thought it would be great to have a nice little portable lab environment where I could deploy a few lightweight VMs and practice managing them with Salt - without impacting any systems that are actually being used for anything. Along the way, I figured I'd leverage [HashiCorp Vagrant](https://www.vagrantup.com/) to create and manage the VMs, which would provide a declarative way to define what the VMs should look like. The VM (or even groups of VMs) would be specified in a single file, and I'd bypass all the tedious steps of creating the virtual hardware, attaching the installation media, installing the OS, and performing the initial configuration. Vagrant will help me build up, destroy, and redeploy a development environment in a simple and repeatable way.
Also, because I'm a bit of a sadist, I wanted to do this all on my new [Framework Chromebook](https://frame.work/laptop-chromebook-12-gen-intel). I might as well put my 32GB of RAM to good use, right?
It took a bit of fumbling, but this article describes what it took to get a Vagrant-powered VM up and running in the [Linux Development Environment](https://chromeos.dev/en/linux) on my Chromebook (which is currently running ChromeOS v111 beta).
### Install the prerequisites
There are are a few packages which need to be installed before we can move on to the Vagrant-specific stuff. It's quite possible that these are already on your system.... but if they *aren't* already present you'll have a bad problem[^problem].
```shell
sudo apt update
sudo apt install \
build-essential \
gpg \
lsb-release \
wget
```
[^problem]: and [will not go to space today](https://xkcd.com/1133/).
I'll be configuring Vagrant to use [`libvirt`](https://libvirt.org/) to interface with the [Kernel Virtual Machine (KVM)](https://www.linux-kvm.org/page/Main_Page) virtualization solution (rather than something like VirtualBox that would bring more overhead) so I'll need to install some packages for that as well:
```shell
sudo apt install virt-manager libvirt-dev
```
And to avoid having to `sudo` each time I interact with `libvirt` I'll add myself to that group:
```shell
sudo gpasswd -a $USER libvirt ; newgrp libvirt
```
And to avoid [this issue](https://github.com/virt-manager/virt-manager/issues/333) I'll make a tweak to the `qemu.conf` file:
```shell
echo "remember_owner = 0" | sudo tee -a /etc/libvirt/qemu.conf
sudo systemctl restart libvirtd
```
I'm also going to use `rsync` to share a [synced folder](https://developer.hashicorp.com/vagrant/docs/synced-folders/basic_usage) between the host and the VM guest so I'll need to make sure that's installed too:
```shell
sudo apt install rsync
```
### Install Vagrant
With that out of the way, I'm ready to move on to the business of installing Vagrant. I'll start by adding the HashiCorp repository:
```shell
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
```
I'll then install the Vagrant package:
```shell
sudo apt update
sudo apt install vagrant
```
I also need to install the [`vagrant-libvirt` plugin](https://github.com/vagrant-libvirt/vagrant-libvirt) so that Vagrant will know how to interact with `libvirt`:
```shell
vagrant plugin install vagrant-libvirt
```
### Create a lightweight VM
Now I can get to the business of creating my first VM with Vagrant!
Vagrant VMs are distributed as Boxes, and I can browse some published Boxes at [app.vagrantup.com/boxes/search?provider=libvirt](https://app.vagrantup.com/boxes/search?provider=libvirt) (applying the `provider=libvirt` filter so that I only see Boxes which will run on my chosen virtualization provider). For my first VM, I'll go with something light and simple: [`generic/alpine38`](https://app.vagrantup.com/generic/boxes/alpine38).
So I'll create a new folder to contain the Vagrant configuration:
```shell
mkdir vagrant-alpine
cd vagrant-alpine
```
And since I'm referencing a Vagrant Box which is published on Vagrant Cloud, downloading the config is as simple as:
```shell
vagrant init generic/alpine38
```
That lets me know that
```text
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
```
Before I `vagrant up` the joint, I do need to make a quick tweak to the default Vagrantfile, which is what tells Vagrant how to configure the VM. By default, Vagrant will try to create a synced folder using NFS and will throw a nasty error when that (inevitably[^inevitable]) fails. So I'll open up the Vagrantfile to review and edit it:
```shell
vim Vagrantfile
```
Most of the default Vagrantfile is commented out. Here's the entirey of the configuration *without* the comments:
```ruby
Vagrant.configure("2") do |config|
config.vm.box = "generic/alpine38"
end
```
There's not a lot there, is there? Well I'm just going to add these two lines somewhere between the `Vagrant.configure()` and `end` lines:
```ruby
config.nfs.verify_installed = false
config.vm.synced_folder '.', '/vagrant', type: 'rsync'
```
The first line tells Vagrant not to bother checking to see if NFS is installed, and will use `rsync` to share the local directory with the VM guest, where it will be mounted at `/vagrant`.
So here's the full Vagrantfile (sans-comments[^magic], again):
```ruby
Vagrant.configure("2") do |config|
config.vm.box = "generic/alpine38"
config.nfs.verify_installed = false
config.vm.synced_folder '.', '/vagrant', type: 'rsync'
end
```
With that, I'm ready to fire up this VM with `vagrant up`! Vagrant will look inside `Vagrantfile` to see the config, pull down the `generic/alpine38` Box from Vagrant Cloud, boot the VM, configure it so I can SSH in to it, and mount the synced folder:
```shell
; vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Box 'generic/alpine38' could not be found. Attempting to find and install...
default: Box Provider: libvirt
default: Box Version: >= 0
==> default: Loading metadata for box 'generic/alpine38'
default: URL: https://vagrantcloud.com/generic/alpine38
==> default: Adding box 'generic/alpine38' (v4.2.12) for provider: libvirt
default: Downloading: https://vagrantcloud.com/generic/boxes/alpine38/versions/4.2.12/providers/libvirt.box
default: Calculating and comparing box checksum...
==> default: Successfully added box 'generic/alpine38' (v4.2.12) for 'libvirt'!
==> default: Uploading base box image as volume into Libvirt storage...
[...]
==> default: Waiting for domain to get an IP address...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 192.168.121.41:22
default: SSH username: vagrant
default: SSH auth method: private key
[...]
default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Rsyncing folder: /home/john/projects/vagrant-alpine/ => /vagrant
```
And then I can use `vagrant ssh` to log in to the new VM:
```shell
; vagrant ssh
alpine38:~$ cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.8.5
PRETTY_NAME="Alpine Linux v3.8"
HOME_URL="http://alpinelinux.org"
BUG_REPORT_URL="http://bugs.alpinelinux.org"
```
I can also verify that the synced folder came through as expected:
```shell
alpine38:~$ ls -l /vagrant
total 4
-rw-r--r-- 1 vagrant vagrant 3117 Feb 20 15:51 Vagrantfile
```
Once I'm finished poking at this VM, shutting it down is as easy as:
```shell
vagrant halt
```
And if I want to clean up and remove all traces of the VM, that's just:
```shell
vagrant destroy
```
[^inevitable]: NFS doesn't work properly from within an LXD container, like the ChromeOS Linux development environment.
[^magic]: Through the magic of `egrep -v "^\s*(#|$)" $file`.
### Create a heavy VM, as a treat
Having proven to myself that Vagrant does work on a Chromebook, let's see how it does with a slightly-heavier VM.... like [Windows 11](https://app.vagrantup.com/oopsme/boxes/windows11-22h2).
{{% notice info "Space Requirement" %}}
Windows 11 makes for a pretty hefty VM which will require significant storage space. My Chromebook's Linux environment ran out of storage space the first time I attempted to deploy this guy. Fortunately ChromeOS makes it easy to allocate more space to Linux (**Settings > Advanced > Developers > Linux development environment > Disk size**). You'll probably need at least 30GB free to provision this VM.
{{% /notice %}}
Again, I'll create a new folder to hold the Vagrant configuration and do a `vagrant init`:
```shell
mkdir vagrant-win11
cd vagrant-win11
vagrant init oopsme/windows11-22h2
```
And, again, I'll edit the Vagrantfile before starting the VM. This time, though, I'm adding a few configuration options to tell `libvirt` that I'd like more compute resources than the default 1 CPU and 512MB RAM[^ram]:
```ruby
Vagrant.configure("2") do |config|
config.vm.box = "oopsme/windows11-22h2"
config.vm.provider :libvirt do |libvirt|
libvirt.cpus = 4
libvirt.memory = 4096
end
end
```
[^ram]: Note here that `libvirt.memory` is specified in MB. Windows 11 boots happily with 4096 MB of RAM.... and somewhat less so with just 4 MB. *Ask me how I know...*
Now it's time to bring it up. This one's going to take A While as it syncs the ~12GB Box first.
```shell
vagrant up
```
Eventually it should spit out that lovely **Machine booted and ready!** message and I can log in! I *can* do a `vagrant ssh` again to gain a shell in the Windows environment, but I'll probably want to interact with those sweet sweet graphics. That takes a little bit more effort.
First, I'll use `virsh -c qemu:///system list` to see the running VM(s):
```shell
; virsh -c qemu:///system list
Id Name State
---------------------------------------
10 vagrant-win11_default running
```
Then I can tell `virt-viewer` that I'd like to attach a session there:
```shell
virt-viewer -c qemu:///system -a vagrant-win11_default
```
I log in with the default password `vagrant`, and I'm in Windows 11 land!
![Windows 11 running on a Chromebook!](win-11-vm.png)
### Next steps
Well that about does it for a proof-of-concept. My next steps will be exploring [multi-machine Vagrant environments](https://developer.hashicorp.com/vagrant/docs/multi-machine) to create a portable lab environment including machines running several different operating systems so that I can learn how to manage them effectively with Salt. It should be fun!

Binary file not shown.

After

Width:  |  Height:  |  Size: 82 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 918 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 476 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 212 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 176 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 310 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

View file

@ -0,0 +1,434 @@
---
series: Projects
date: "2021-06-28T00:00:00Z"
thumbnail: 2xe34VJym.png
usePageBundles: true
lastmod: "2021-09-17"
tags:
- docker
- linux
- cloud
- containers
- chat
title: Federated Matrix Server (Synapse) on Oracle Cloud's Free Tier
---
I've heard a lot lately about how generous [Oracle Cloud's free tier](https://www.oracle.com/cloud/free/) is, particularly when [compared with the free offerings](https://github.com/cloudcommunity/Cloud-Service-Providers-Free-Tier-Overview) from other public cloud providers. Signing up for an account was fairly straight-forward, though I did have to wait a few hours for an actual human to call me on an actual telephone to verify my account. Once in, I thought it would be fun to try building my own [Matrix](https://matrix.org/) homeserver to really benefit from the network's decentralized-but-federated model for secure end-to-end encrypted communications.
There are two primary projects for Matrix homeservers: [Synapse](https://github.com/matrix-org/synapse/) and [Dendrite](https://github.com/matrix-org/dendrite). Dendrite is the newer, more efficient server, but it's not quite feature complete. I'll be using Synapse for my build to make sure that everything works right off the bat, and I will be running the server in a Docker container to make it (relatively) easy to replace if I feel more comfortable about Dendrite in the future.
As usual, it took quite a bit of fumbling about before I got everything working correctly. Here I'll share the steps I used to get up and running.
### Instance creation
Getting a VM spun up on Oracle Cloud was a pretty simple process. I logged into my account, navigated to *Menu -> Compute -> Instances*, and clicked on the big blue **Create Instance** button.
![Create Instance](8XAB60aqk.png)
I'll be hosting this for my `bowdre.net` domain, so I start by naming the instance accordingly: `matrix.bowdre.net`. Naming it isn't strictly necessary, but it does help with keeping track of things. The instance defaults to using an Oracle Linux image. I'd rather use an Ubuntu one for this, simply because I was able to find more documentation on getting Synapse going on Debian-based systems. So I hit the **Edit** button next to *Image and Shape*, select the **Change Image** option, pick **Canonical Ubuntu** from the list of available images, and finally click **Select Image** to confirm my choice.
![Image Selection](OSbsiOw8E.png)
This will be an Ubuntu 20.04 image running on a `VM.Standard.E2.1.Micro` instance, which gets a single AMD EPYC 7551 CPU with 2.0GHz base frequency and 1GB of RAM. It's not much, but it's free - and it should do just fine for this project.
I can leave the rest of the options as their defaults, making sure that the instance will be allotted a public IPv4 address.
![Other default selections](Ki0z1C3g.png)
Scrolling down a bit to the *Add SSH Keys* section, I leave the default **Generate a key pair for me** option selected, and click the very-important **Save Private Key** button to download the private key to my computer so that I'll be able to connect to the instance via SSH.
![Download Private Key](dZkZUIFum.png)
Now I can finally click the blue **Create Instance** button at the bottom of the screen, and just wait a few minutes for it to start up. Once the status shows a big green "Running" square, I'm ready to connect! I'll copy the listed public IP and make a note of the default username (`ubuntu`). I can then plug the IP, username, and the private key I downloaded earlier into my SSH client (the [Secure Shell extension](https://chrome.google.com/webstore/detail/secure-shell/iodihamcpbpeioajjeobimgagajmlibd) for Google Chrome since I'm doing this from my Pixelbook), and log in to my new VM in The Cloud.
![Logged in!](5PD1H7b1O.png)
### DNS setup
According to [Oracle's docs](https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingpublicIPs.htm), the public IP assigned to my instance is mine until I terminate the instance. It should even remain assigned if I stop or restart the instance, just as long as I don't delete the virtual NIC attached to it. So I'll skip the [`ddclient`-based dynamic DNS configuration I've used in the past](/bitwarden-password-manager-self-hosted-on-free-google-cloud-instance#configure-dynamic-dns) and instead go straight to my registrar's DNS management portal and create a new `A` record for `matrix.bowdre.net` with the instance's public IP.
While I'm managing DNS, it might be good to take a look at the requirements for [federating my new server](https://github.com/matrix-org/synapse/blob/master/docs/federate.md#setting-up-federation) with the other Matrix servers out there. I'd like for users identities on my server to be identified by the `bowdre.net` domain (`@user:bowdre.net`) rather than the full `matrix.bowdre.net` FQDN (`@user:matrix.bowdre.net` is kind of cumbersome). The standard way to do this to leverage [`.well-known` delegation](https://github.com/matrix-org/synapse/blob/master/docs/delegate.md#well-known-delegation), where the URL at `http://bowdre.net/.well-known/matrix/server` would return a JSON structure telling other Matrix servers how to connect to mine:
```json
{
"m.server": "matrix.bowdre.net:8448"
}
```
I don't *currently* have another server already handling requests to `bowdre.net`, so for now I'll add another `A` record with the same public IP address to my DNS configuration. Requests for both `bowdre.net` and `matrix.bowdre.net` will reach the same server instance, but those requests will be handled differently. More on that later.
An alternative to this `.well-known` delegation would be to use [`SRV` DNS record delegation](https://github.com/matrix-org/synapse/blob/master/docs/delegate.md#srv-dns-record-delegation) to accomplish the same thing. I'd create an `SRV` record for `_matrix._tcp.bowdre.net` with the data `0 10 8448 matrix.bowdre.net` (priority=`0`, weight=`10`, port=`8448`, target=`matrix.bowdre.net`) which would again let other Matrix servers know where to send the federation traffic for my server. This approach has an advantage of not needing to make any changes on the `bowdre.net` web server, but it would require the delegated `matrix.bowdre.net` server to *also* [return a valid certificate for `bowdre.net`](https://matrix.org/docs/spec/server_server/latest#:~:text=If%20the%20/.well-known%20request%20resulted,present%20a%20valid%20certificate%20for%20%3Chostname%3E.). Trying to get a Let's Encrypt certificate for a server name that doesn't resolve authoritatively in DNS sounds more complicated than I want to get into with this project, so I'll move forward with my plan to use the `.well-known` delegation instead.
But first, I need to make sure that the traffic reaches the server to begin with.
### Firewall configuration
Synapse listens on port `8008` for connections from messaging clients, and typically uses port `8448` for federation traffic from other Matrix servers. Rather than expose those ports directly, I'm going to put Synapse behind a reverse proxy on HTTPS port `443`. I'll also need to allow inbound traffic HTTP port `80` for ACME certificate challenges. I've got two firewalls to contend with: the Oracle Cloud one which blocks traffic from getting into my virtual cloud network, and the host firewall running inside the VM.
I'll tackle the cloud firewall first. From the page showing my instance details, I click on the subnet listed under the *Primary VNIC* heading:
![Click on subnet](lBjINolYq.png)
I then look in the *Security Lists* section and click on the Default Security List:
![Click on default security list](nnQ7aQrpm.png)
The *Ingress Rules* section lists the existing inbound firewall exceptions, which by default is basically just SSH. I click on **Add Ingress Rules** to create a new one.
![Ingress rules](dMPHvLHkH.png)
I want this to apply to traffic from any source IP so I enter the CIDR `0.0.0.0/0`, and I enter the *Destination Port Range* as `80,443`. I also add a brief description and click **Add Ingress Rules**.
![Adding an ingress rule](2fbKJc5Y6.png)
Success! My new ingress rules appear at the bottom of the list.
![New rules added](s5Y0rycng.png)
That gets traffic from the internet and to my instance, but the OS is still going to drop the traffic at its own firewall. I'll need to work with `iptables` to change that. (You typically use `ufw` to manage firewalls more easily on Ubuntu, but it isn't included on this minimal image and seemed to butt heads with `iptables` when I tried adding it. I eventually decided it was better to just interact with `iptables` directly). I'll start by listing the existing rules on the `INPUT` chain:
```
$ sudo iptables -L INPUT --line-numbers
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
2 ACCEPT icmp -- anywhere anywhere
3 ACCEPT all -- anywhere anywhere
4 ACCEPT udp -- anywhere anywhere udp spt:ntp
5 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
6 REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
```
Note the `REJECT all` statement at line `6`. I'll need to insert my new `ACCEPT` rules for ports `80` and `443` above that implicit deny all:
```
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT
```
And then I'll confirm that the order is correct:
```
$ sudo iptables -L INPUT --line-numbers
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
2 ACCEPT icmp -- anywhere anywhere
3 ACCEPT all -- anywhere anywhere
4 ACCEPT udp -- anywhere anywhere udp spt:ntp
5 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
6 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https
7 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
8 REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
```
I can use `nmap` running from my local Linux environment to confirm that I can now reach those ports on the VM. (They're still "closed" since nothing is listening on the ports yet, but the connections aren't being rejected.)
```
$ nmap -Pn matrix.bowdre.net
Starting Nmap 7.70 ( https://nmap.org ) at 2021-06-27 12:49 CDT
Nmap scan report for matrix.bowdre.net(150.136.6.180)
Host is up (0.086s latency).
Other addresses for matrix.bowdre.net (not scanned): 2607:7700:0:1d:0:1:9688:6b4
Not shown: 997 filtered ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp closed http
443/tcp closed https
Nmap done: 1 IP address (1 host up) scanned in 8.44 seconds
```
Cool! Before I move on, I'll be sure to make the rules persistent so they'll be re-applied whenever `iptables` starts up:
Make rules persistent:
```
$ sudo netfilter-persistent save
run-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save
run-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save
```
### Reverse proxy setup
I had initially planned on using `certbot` to generate Let's Encrypt certificates, and then reference the certs as needed from an `nginx` or Apache reverse proxy configuration. While researching how the [proxy would need to be configured to front Synapse](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md), I found this sample `nginx` configuration:
```conf
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
# For the federation port
listen 8448 ssl http2 default_server;
listen [::]:8448 ssl http2 default_server;
server_name matrix.example.com;
location ~* ^(\/_matrix|\/_synapse\/client) {
proxy_pass http://localhost:8008;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
# Nginx by default only allows file uploads up to 1M in size
# Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
client_max_body_size 50M;
}
}
```
And this sample Apache one:
```conf
<VirtualHost *:443>
SSLEngine on
ServerName matrix.example.com
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
AllowEncodedSlashes NoDecode
ProxyPreserveHost on
ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix
ProxyPass /_synapse/client http://127.0.0.1:8008/_synapse/client nocanon
ProxyPassReverse /_synapse/client http://127.0.0.1:8008/_synapse/client
</VirtualHost>
<VirtualHost *:8448>
SSLEngine on
ServerName example.com
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
AllowEncodedSlashes NoDecode
ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix
</VirtualHost>
```
I also found this sample config for another web server called [Caddy](https://caddyserver.com):
```
matrix.example.com {
reverse_proxy /_matrix/* http://localhost:8008
reverse_proxy /_synapse/client/* http://localhost:8008
}
example.com:8448 {
reverse_proxy http://localhost:8008
}
```
One of these looks much simpler than the other two. I'd never heard of Caddy so I did some quick digging, and I found that it would actually [handle the certificates entirely automatically](https://caddyserver.com/docs/automatic-https) - in addition to having a much easier config. [Installing Caddy](https://caddyserver.com/docs/install#debian-ubuntu-raspbian) wasn't too bad, either:
```sh
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo apt-key add -
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy
```
Then I just need to put my configuration into the default `Caddyfile`, including the required `.well-known` delegation piece from earlier.
```
$ sudo vi /etc/caddy/Caddyfile
matrix.bowdre.net {
reverse_proxy /_matrix/* http://localhost:8008
reverse_proxy /_synapse/client/* http://localhost:8008
}
bowdre.net {
route {
respond /.well-known/matrix/server `{"m.server": "matrix.bowdre.net:443"}`
redir https://virtuallypotato.com
}
}
```
There's a lot happening in that 11-line `Caddyfile`, but it's not complicated by any means. The `matrix.bowdre.net` section is pretty much exactly yanked from the sample config, and it's going to pass any requests that start like `matrix.bowdre.net/_matrix/` or `matrix.bowdre.net/_synapse/client/` through to the Synapse server listening locally on port `8008`. Caddy will automatically request and apply a Let's Encrypt or ZeroSSL cert for any server names spelled out in the config - very slick!
I set up the `bowdre.net` section to return the appropriate JSON string to tell other Matrix servers to connect to `matrix.bowdre.net` on port `443` (so that I don't have to open port `8448` through the firewalls), and to redirect all other traffic to one of my favorite technical blogs (maybe you've heard of it?). I had to wrap the `respond` and `redir` directives in a [`route { }` block](https://caddyserver.com/docs/caddyfile/directives/route) because otherwise Caddy's [implicit precedence](https://caddyserver.com/docs/caddyfile/directives#directive-order) would execute the redirect for *all* traffic and never hand out the necessary `.well-known` data.
(I wouldn't need that section at all if I were using a separate web server for `bowdre.net`; instead, I'd basically just add that `respond /.well-known/matrix/server` line to that other server's config.)
Now to enable the `caddy` service, start it, and restart it so that it loads the new config:
```
sudo systemctl enable caddy
sudo systemctl start caddy
sudo systemctl restart caddy
```
If I repeat my `nmap` scan from earlier, I'll see that the HTTP and HTTPS ports are now open. The server still isn't actually serving anything on those ports yet, but at least it's listening.
```
$ nmap -Pn matrix.bowdre.net
Starting Nmap 7.70 ( https://nmap.org ) at 2021-06-27 13:44 CDT
Nmap scan report for matrix.bowdre.net (150.136.6.180)
Host is up (0.034s latency).
Not shown: 997 filtered ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
443/tcp open https
Nmap done: 1 IP address (1 host up) scanned in 5.29 seconds
```
Browsing to `https://matrix.bowdre.net` shows a blank page - but a valid and trusted certificate that I did absolutely nothing to configure!
![Valid cert!](GHVqVOTAE.png)
The `.well-known` URL also returns the expected JSON:
![.well-known](6IRPHhr6u.png)
And trying to hit anything else at `https://bowdre.net` brings me right back here.
And again, the config to do all this (including getting valid certs for two server names!) is just 11 lines long. Caddy is seriously and magically cool.
Okay, let's actually serve something up now.
### Synapse installation
#### Docker setup
Before I can get on with [deploying Synapse in Docker](https://hub.docker.com/r/matrixdotorg/synapse), I first need to [install Docker](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) on the system:
```sh
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io
```
I'll also [install Docker Compose](https://docs.docker.com/compose/install/#install-compose):
```sh
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
```
And I'll add my `ubuntu` user to the `docker` group so that I won't have to run every docker command with `sudo`:
```
sudo usermod -G docker -a ubuntu
```
I'll log out and back in so that the membership change takes effect, and then test both `docker` and `docker-compose` to make sure they're working:
```
$ docker --version
Docker version 20.10.7, build f0df350
$ docker-compose --version
docker-compose version 1.29.2, build 5becea4c
```
#### Synapse setup
Now I'll make a place for the Synapse installation to live, including a `data` folder that will be mounted into the container:
```
sudo mkdir -p /opt/matrix/synapse/data
cd /opt/matrix/synapse
```
And then I'll create the compose file to define the deployment:
```yaml
$ sudo vi docker-compose.yml
services:
synapse:
container_name: "synapse"
image: "matrixdotorg/synapse"
restart: "unless-stopped"
ports:
- "127.0.0.1:8008:8008"
volumes:
- "./data/:/data/"
```
Before I can fire this up, I'll need to generate an initial configuration as [described in the documentation](https://hub.docker.com/r/matrixdotorg/synapse). Here I'll specify the server name that I'd like other Matrix servers to know mine by (`bowdre.net`):
```sh
$ docker run -it --rm \
-v "/opt/matrix/synapse/data:/data" \
-e SYNAPSE_SERVER_NAME=bowdre.net \
-e SYNAPSE_REPORT_STATS=yes \
matrixdotorg/synapse generate
Unable to find image 'matrixdotorg/synapse:latest' locally
latest: Pulling from matrixdotorg/synapse
69692152171a: Pull complete
66a3c154490a: Pull complete
3e35bdfb65b2: Pull complete
f2c4c4355073: Pull complete
65d67526c337: Pull complete
5186d323ad7f: Pull complete
436afe4e6bba: Pull complete
c099b298f773: Pull complete
50b871f28549: Pull complete
Digest: sha256:5ccac6349f639367fcf79490ed5c2377f56039ceb622641d196574278ed99b74
Status: Downloaded newer image for matrixdotorg/synapse:latest
Creating log config /data/bowdre.net.log.config
Generating config file /data/homeserver.yaml
Generating signing key file /data/bowdre.net.signing.key
A config file has been generated in '/data/homeserver.yaml' for server name 'bowdre.net'. Please review this file and customise it to your needs.
```
As instructed, I'll use `sudo vi data/homeserver.yaml` to review/modify the generated config. I'll leave
```yaml
server_name: "bowdre.net"
```
since that's how I'd like other servers to know my server, and I'll uncomment/edit in:
```yaml
public_baseurl: https://matrix.bowdre.net
```
since that's what users (namely, me) will put into their Matrix clients to connect.
And for now, I'll temporarily set:
```yaml
enable_registration: true
```
so that I can create a user account without fumbling with the CLI. I'll be sure to set `enable_registration: false` again once I've registered the account(s) I need to have on my server. The instance has limited resources so it's probably not a great idea to let just anybody create an account on it.
There are a bunch of other useful configurations that can be made here, but these will do to get things going for now.
Time to start it up:
```
$ docker-compose up -d
Creating network "synapse_default" with the default driver
Creating synapse ... done
```
And use `docker ps` to confirm that it's running:
```
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
573612ec5735 matrixdotorg/synapse "/start.py" 25 seconds ago Up 23 seconds (healthy) 8009/tcp, 127.0.0.1:8008->8008/tcp, 8448/tcp synapse
```
### Testing
And I can point my browser to `https://matrix.bowdre.net/_matrix/static/` and see the Matrix landing page:
![Synapse is running!](-9apQIUci.png)
Before I start trying to connect with a client, I'm going to plug the server address in to the [Matrix Federation Tester](https://federationtester.matrix.org/) to make sure that other servers will be able to talk to it without any problems:
![Good to go](xqOt3SydX.png)
And I can view the JSON report at the bottom of the page to confirm that it's correctly pulling my `.well-known` delegation:
```json
{
"WellKnownResult": {
"m.server": "matrix.bowdre.net:443",
"CacheExpiresAt": 0
},
```
Now I can fire up my [Matrix client of choice](https://element.io/get-started)), specify my homeserver using its full FQDN, and [register](https://app.element.io/#/register) a new user account:
![image.png](2xe34VJym.png)
(Once my account gets created, I go back to edit `/opt/matrix/synapse/data/homeserver.yaml` again and set `enable_registration: false`, then fire a `docker-compose restart` command to restart the Synapse container.)
### Wrap-up
And that's it! I now have my own Matrix server, and I can use my new account for secure chats with Matrix users on any other federated homeserver. It works really well for directly messaging other individuals, and also for participating in small group chats. The server *does* kind of fall on its face if I try to join a massively-populated (like 500+ users) room, but I'm not going to complain about that too much on a free-tier server.
All in, I'm pretty pleased with how this little project turned out, and I learned quite a bit along the way. I'm tremendously impressed by Caddy's power and simplicity, and I look forward to using it more in future projects.
### Update: Updating
After a while, it's probably a good idea to update both the Ubntu server and the Synapse container running on it. Updating the server itself is as easy as:
```sh
sudo apt update
sudo apt upgrade
# And, if needed:
sudo reboot
```
Here's what I do to update the container:
```sh
# Move to the working directory
cd /opt/matrix/synapse
# Pull a new version of the synapse image
docker-compose pull
# Stop the container
docker-compose down
# Start it back up without the old version
docker-compose up -d --remove-orphans
# Periodically remove the old docker images
docker image prune
```

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

View file

@ -0,0 +1,53 @@
---
series: Tips
date: "2020-09-13T08:34:30Z"
usePageBundles: true
tags:
- linux
- shell
- logs
- regex
title: Finding the most popular IPs in a log file
---
I found myself with a sudden need for parsing a Linux server's logs to figure out which host(s) had been slamming it with an unexpected burst of traffic. Sure, there are proper log analysis tools out there which would undoubtedly make short work of this but none of those were installed on this hardened system. So this is what I came up with.
### Find IP-ish strings
This will get you all occurrences of things which look vaguely like IPv4 addresses:
```shell
grep -o -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ACCESS_LOG.TXT
```
(It's not a perfect IP address regex since it would match things like `987.654.321.555` but it's close enough for my needs.)
### Filter out `localhost`
The log likely include a LOT of traffic to/from `127.0.0.1` so let's toss out `localhost` by piping through `grep -v "127.0.0.1"` (`-v` will do an inverse match - only return results which *don't* match the given expression):
```shell
grep -o -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ACCESS_LOG.TXT | grep -v "127.0.0.1"
```
### Count up the duplicates
Now we need to know how many times each IP shows up in the log. We can do that by passing the output through `uniq -c` (`uniq` will filter for unique entries, and the `-c` flag will return a count of how many times each result appears):
```shell
grep -o -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ACCESS_LOG.TXT | grep -v "127.0.0.1" | uniq -c
```
### Sort the results
We can use `sort` to sort the results. `-n` tells it sort based on numeric rather than character values, and `-r` reverses the list so that the larger numbers appear at the top:
```shell
grep -o -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ACCESS_LOG.TXT | grep -v "127.0.0.1" | uniq -c | sort -n -r
```
### Top 5
And, finally, let's use `head -n 5` to only get the first five results:
```shell
grep -o -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ACCESS_LOG.TXT | grep -v "127.0.0.1" | uniq -c | sort -n -r | head -n 5
```
### Bonus round!
You know how old log files get rotated and compressed into files like `logname.1.gz`? I *very* recently learned that there are versions of the standard Linux text manipulation tools which can work directly on compressed log files, without having to first extract the files. I'd been doing things the hard way for years - no longer, now that I know about `zcat`, `zdiff`, `zgrep`, and `zless`!
So let's use a `for` loop to iterate through 20 of those compressed logs, and use `date -r [filename]` to get the timestamp for each log as we go:
```bash
for i in {1..20}; do date -r ACCESS_LOG.$i.gz; zgrep -o -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' \ACCESS_LOG.log.$i.gz | grep -v "127.0.0.1" | uniq -c | sort -n -r | head -n 5; done
```
Nice!

Binary file not shown.

After

Width:  |  Height:  |  Size: 95 KiB

View file

@ -0,0 +1,28 @@
---
date: "2020-10-07T08:34:30Z"
thumbnail: MnmMuA0HC.png
usePageBundles: true
tags:
- windows
- linux
- wsl
- vpn
title: Fixing WSL2 connectivity when connected to a VPN with wsl-vpnkit
toc: false
---
I was pretty excited to get [WSL2 and Docker working on my Windows 10 1909](/docker-on-windows-10-with-wsl2) laptop a few weeks ago, but I quickly encountered a problem: WSL2 had no network connectivity when connected to my work VPN.
Well, that's not *entirely* true; Docker worked just fine, but nothing else could talk to anything outside of the WSL environment. I found a few open issues for this problem in the [WSL2 Github](https://github.com/microsoft/WSL/issues?q=is%3Aissue+is%3Aopen+VPN) with suggested workarounds including modifying Windows registry entries, adjusting the metrics assigned to various virtual network interfaces within Windows, and manually setting DNS servers in `/etc/resolv.conf`. None of these worked for me.
I eventually came across a solution [here](https://github.com/sakai135/wsl-vpnkit) which did the trick. This takes advantage of the fact that Docker for Windows is already utilizing `vpnkit` for connectivity - so you may also want to be sure Docker Desktop is configured to start at login.
The instructions worked well for me so I won't rehash them all here. When it came time to modify my `/etc/resolv.conf` file, I added in two of the internal DNS servers followed by the IP for my home router's DNS service. This allows me to use WSL2 both on and off the corporate network without having to reconfigure things.
All I need to do now is execute `sudo ./wsl-vpnkit` and leave that running in the background when I need to use WSL while connected to the corporate VPN.
![Successful connection via wsl-vpnkit](MnmMuA0HC.png)
Whew! Okay, back to work.

Binary file not shown.

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 88 KiB

Some files were not shown because too many files have changed in this diff Show more