Mac Server Reliability Scripts

Posted on by Mark Boszko

NOTE: This is based on a GitHub repo at https://github.com/swizzlevixen/Mac-Server-Reliability which may contain updates from what is written here. Please Check my GitHub, or comment here, if you encounter any errors.

This is a collection of helpful notes, and a suite of scripts written for my Mac mini M2 server, that help with server reliability, and notifications of errors. This server currently runs Plex, DEVONthink Server, Sonos, and Apple Music.

The scripts in this document have some sensitive information replaced with placeholders, for privacy. Otherwise they should be an accurate representation of what is running on my server.

Automatically log in on startup

Due to the nature of the apps that I'm using on this server, the Mac needs to run with a user logged in. This is accomplished with System Settings > Users & Groups > Automatically log in as… and select the user admin. Note that FileVault must be disabled for automatic login to be available. Yes, that's less secure, but this Mac lives in an equipment rack in my home, and if someone we don't trust has physical access to that machine, we have bigger problems.

Energy Settings

We want this machine to always be running, and recover after a power failure or other interruption. So I have these settings:

  • System Settings > Login Password > Automatically login after a restart: on
  • System Settings > Energy > Prevent automatic sleeping when the display is off: on
  • System Settings > Energy > Wake for network access: on
  • System Settings > Energy > Start up automatically after a power failure: on
  • Lock Screen > (the top group of three settings): Never

Login items

You can easily set apps and network drives to launch when the user logs in. Go to System Settings > General > Login Items & Extensions and you can either drag items into the Open at Login list, or use the + button at the bottom of the list.

Currently in the list:

  • CCC Dashboard (Carbon Copy Cloner, for backups)
  • scandocs network drive, from the Synology mynas NAS
  • plexmedia network drive, from the Synology mynas NAS
  • Screens Connect (for remote access)
  • Startup Apps and Databases (app version of the AppleScript, see below)

Startup Apps and Databases

Of course, there are more apps that we need to be running on this server, but I was running into two issues when putting them all in the Login Items list — this M2 Mac mini is so fast that the apps would start up before the network drives were connected, and would not have the necessary files available. The second issue, we'll get to in a moment.

To solve this, I wrote the Startup Apps and Databases AppleScript. This script expects to launch as a Login Item, waits to make sure that the network drives have mounted, and then launches the server apps one by one, giving a few seconds between each one, to make sure they get a chance to start up smoothly.

(*
Startup Apps and Databases
Copyright © 2024-2025 Mark Boszko
*)

-- Delay interval to wait between steps, in seconds
set theInterval to 5

-- Log start
do shell script "zsh /Applications/log-event.sh \"Startup Apps and Databases: running...\""

(*
*************************
Synology network drives
*************************
- Make sure the network drives are mounted
- Once they are, start the apps that are dependent on the files
*)

-- Grab the server drive icon
set GenericFileServerIcon to POSIX file "/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericFileServerIcon.icns"

-- delay after boot, then see if the network drives are mounted
set diskMounted to false
repeat until diskMounted is true
    -- Check if the network drive is mounted
    tell application "System Events" to set diskNames to name of every disk
    if "plexmedia" is in diskNames and "scandocs" is in diskNames then
        set diskMounted to true
        tell me to activate
        display dialog "plexmedia and scandocs network drives are mounted.
Launching apps dependent on availability." buttons {"Cancel"} default button 1 with icon GenericFileServerIcon giving up after theInterval
    else
        -- Try to mount it again
        tell application "Finder"
            mount volume "smb://mynas._smb._tcp.local/plexmedia"
            mount volume "smb://mynas._smb._tcp.local/scandocs"
        end tell
        tell me to activate
        display dialog "Waiting for plexmedia and scandocs network drives to mount.
Retrying in " & theInterval & " seconds" as text buttons {"Cancel"} default button 1 with icon GenericFileServerIcon giving up after theInterval
    end if
end repeat

(*
*************************
Plex Media Server
*************************
- Launch it
*)

-- Grab the app icon
set appIcon to POSIX file "/Applications/Plex Media Server.app/Contents/Resources/Plex.icns"
tell me to activate
display dialog "Launching Plex" as text buttons {"Cancel"} default button 1 with icon appIcon giving up after theInterval
tell application "Plex Media Server"
    activate
end tell

(*
*************************
Sonos
*************************
- Launch it
*)

-- Grab the app icon
set appIcon to POSIX file "/Applications/Sonos.app/Contents/Resources/AppIcon.icns"
tell me to activate
display dialog "Launching Sonos" as text buttons {"Cancel"} default button 1 with icon appIcon giving up after theInterval
tell application "Sonos.app"
    activate
end tell

(*
*************************
Apple Music (with iTunes Match)
*************************
- Launch it
*)

-- Grab the app icon
set appIcon to POSIX file "/System/Applications/Music.app/Contents/Resources/AppIcon.icns"
tell me to activate
display dialog "Launching Music" as text buttons {"Cancel"} default button 1 with icon appIcon giving up after theInterval
tell application "Music"
    activate
end tell

(*
*************************
DEVONthink
*************************
- Launch it
- Open the Databases that I always want to have open in DEVONthink
    - Add any new databases to the list in databaseNames
*)

-- Grab the app icon
set appIcon to POSIX file "/Applications/DEVONthink 3.app/Contents/Resources/DEVONthink 3.icns"
tell me to activate
display dialog "Launching DEVONthink and loading databases" as text buttons {"Cancel"} default button 1 with icon appIcon giving up after theInterval
property databaseFolderPath : "/Users/admin/Databases/"
property databaseNames : {"My Scans", "Shared", "Other Scans"}

tell application id "DNtp"
    activate
    -- Give it a moment to launch
    delay theInterval
    try
        repeat with databaseName in databaseNames
            open database (databaseFolderPath & databaseName & ".dtBase2") as string

            -- open window for record (root of database databaseName)
            -- activate
            delay theInterval
        end repeat

    on error error_message number error_number
        if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
    end try
end tell

--Log completion
do shell script "zsh /Applications/log-event.sh \"Startup Apps and Databases: completed\""
--Send notification to iPhone that services have restarted
do shell script "zsh /Applications/hass-notify-iphone.sh  \"Server Notification\" \"Startup Apps and Databases: completed\""

I could have partially done this as a shell script, as I have several other parts of this server monitoring and setup workflow, but AppleScript allowed me to easily show dialogs with information about what was happening, and allow user interaction to cancel if something is going wrong. It also allows me to easily open the necessary databases in DEVONthink for web sharing.

The shell scripts called by this AppleScript are simple helper scripts to write to a log file, and send a notification theough Home Assistant, and they will be discussed later.

Export AppleScript as an App

To allow for the correct permissions to control everything, when it runs on its own, the script needs to be signed, and that means exporting it as an Application. Any time you make changes to the Startup script, follow these steps:

  1. Open Script Editor
  2. Open Startup Apps and Databases Script.scpt
  3. Edit the script as necessary
  4. Save the script
  5. File > Export, delete Script from the name of the file, and save in the Applications folder, with File Format: Application, and Code Sign: Sign to Run Locally
  6. Replace if it asks
  7. If this is your first time adding the script, use the instructions in the Login Items section above to add it as a Login Item. This exported app version of the script is the one that should run every time the Mac boots.
  8. Run the app once, and allow it to manage the computer when asked. This will be in System Settings > Privacy & Security > App Management
  9. You may also need to give it permission to to write to disk, since it is writing to the log file. It will ask for permission, but this is in System Settings > Privacy & Security > Full Disk Access, if you want to add it manually.
  10. Run it again, to make sure it runs without complaint about permissions
  11. Reboot the Mac and let the script run on startup, to confirm all is well

The second problem

If the Mac reboots abnormally (kernel panic, power loss, etc.), it “helpfully” automatically re-opens all previously running apps, which defeats my carefully crafted startup sequence.

The solution is to close all open apps, and then find the .plist that saves information about which apps are currently running, that the Mac uses to re-open the apps when you reboot. Look for this file:

~/Library/Preferences/ByHost/com.apple.loginwindow.*.plist

The * will be a long random character identifier. Select the file in the Finder, and then File > Get Info (⌘-I) to open the Info window for the file. In the General section, check the Locked box, and close the window. This will prevent macOS from changing the contents of the file, and no other apps will be opened if the Mac reboots abnormally.

If it becomes necessary, you can easily revert this behavior by unchecking the box in the Info window.

Helper Scripts

To give myself a little more information about any potential problems, I have scripts that add events to a log file, or send a notification to my iPhone.

Log Event

This simply logs events into a log file on the admin user's Desktop, for easy reading. This is not meant to substitute for full system logs in troubleshooting a problem, but provide a very narrow list of events which may warrant further investigation. The event description is prefaced with a roughly ISO 8601 style date-time string, for easy reading.

#!/bin/zsh
# log-event.sh
# This script logs the current time, with provided text

# Usage:
# log-event.sh "Event description"

# Path to the log file
log_file="/path/to/log/events.log"

# Log the current time with any provided text
{
    echo "$(date +"%Y-%m-%d %H:%M:%S %Z") - $1"
} >> "$log_file"

Send Notification to iPhone

This integration uses Home Assistant, the idea taken from an article written by Viktor Mukha. I modified it to take both the title and message as arguments.

The bearer token is a Long Term token that you can set up in your user account on Home Assistant, and the mobile_app_id at the end of the API endpoint is specific to the name you have given your mobile device. iphone is the name of my particular phone.

Setting up Home Assistant itself is outside the scope of this document, but you can see the linked article above on how to configure this script for your particular setup.

#!/bin/zsh
# hass-notify-iphone.sh
# Based on code © 2024 Viktor Mukha, from this article:
# https://medium.com/@viktor.mukha/push-notifications-from-bash-script-via-home-assistant-852fa92f60ab

# This script sends a notification via Home Assistant
# using the mobile app integration.

# Usage:
# hass-notify-iphone.sh "Title" "Message"

curl -X POST \
    -H "Authorization: Bearer <LONG_TERM_TOKEN>" \
    -H "Content-Type: application/json" \
    -d "{ \
    \"message\": \"$2\", \
    \"title\": \"$1\" \
    }" \
    http://homeassistant.local:8123/api/services/notify/<MOBILE_APP_NAME>

Log Reboots

For a while, I was having trouble with the server kernel panicking, and so I wanted to make sure that I was notified any time the server rebooted. This is meant to run as a LaunchAgent, once on boot. It differs from the log-event script above, in that it grabs the latest boot time from sysctl for accuracy, instead of relying on the current time when this script runs.

#!/bin/zsh
# log-reboot.sh

# This script logs the most recent boot time to the log file
# and sends a notification to Home Assistant

# Path to the log file
log_file="/path/to/log/events.log"

# Check the time of the last reboot
last_reboot=$(sysctl -n kern.boottime | awk '{print $4}' | sed 's/,//')
{
    echo "$(date -r $last_reboot +"%Y-%m-%d %H:%M:%S %Z") - REBOOT"
} >> "$log_file"

# Send notification to iPhone via Home Assistant
/Applications/hass-notify-iphone.sh "Server Reboot" "Server has rebooted."

The LaunchAgent, com.admin.log-reboot.plist should be placed in the folder ~/Library/LaunchAgents/, and then registered with this command:

launchctl bootstrap gui/<ADMIN_USER_ID> ~/Library/LaunchAgents/com.admin.log-reboot.plist

<ADMIN_USER_ID> can be found by running id -u <USERNAME> for the user which you use.

Monitor Network Drives

This script came about because the curent version of macOS Sequoia has an issue connecting over SMB to some Synology drive shares, and sometimes the drives disconnect unexpectedly. So now I have this script that is registered as a LaunchAgent that runs every 10 seconds, to make sure the drives are still connected, and if not, to log and notify me, reconnect them, and restart any necessary apps that rely on them.

It uses osascript (a command line version of AppleScript) to quit the affected apps, because this allows the apps to quit gracefully, as opposed to just killing them without warning.

#!/bin/zsh
# monitor-network-drives.sh

# This script checks if the network drives are mounted,
# logs & notifies any negative status, and attempts to
# remount drives as needed.

# The script is intended to be run as a LaunchAgent
# on macOS, and is triggered by this LaunchAgent plist:
# ~/Library/LaunchAgents/com.admin.monitor-network-drives.plist
# I have this set to run every 10 seconds, for minimal downtime.

# Check the time of the last reboot
last_reboot=$(sysctl -n kern.boottime | awk '{print $4}' | sed 's/,//')
current_time=$(date +%s)
time_since_reboot=$((current_time - last_reboot))

# If the last reboot was less than a minute ago, exit
# so we don't interfere with the startup AppleScript
if [ "$time_since_reboot" -lt 60 ]; then
    echo "The system was rebooted less than a minute ago. Exiting script."
    exit 0
fi

# Check if the network drive "plexmedia" is mounted
if ! mount | grep "plexmedia" > /dev/null; then
    plexmediaMounted=false
    # Log the status
    /Applications/log-event.sh "network drive unmounted: plexmedia"
fi

# Check if the network drive "scandocs" is mounted
if ! mount | grep "scandocs" > /dev/null; then
    scandocsMounted=false
    # Log the status
    /Applications/log-event.sh "network drive unmounted: scandocs"
fi

# If either network drive is not mounted, Send notification to HASS
if [ "$plexmediaMounted" = false ] || [ "$scandocsMounted" = false ]; then
    theMessage="Synology network drive(s) unmounted:"
    if [ "$plexmediaMounted" = false ]; then
        theMessage="$theMessage\n• plexmedia"
    fi
    if [ "$scandocsMounted" = false ]; then
        theMessage="$theMessage\n• scandocs"
    fi
    echo "One or more network drives are not mounted. Sending notification to HASS..."
    /Applications/hass-notify-callisto.sh "Skyfall Error" "$theMessage"
fi

# If the network drive "plexmedia" is not mounted,
# close dependent apps and re-run the Startup script.
# Doing this check before 'scandocs' because if 'plexmedia' is down,
# there's a good chance 'scandocs' is down as well,
# and will also be re-mounted by the Startup script.
if [ "$plexmediaMounted" = false ]; then
    echo "Network drive "plexmedia" is not mounted. Closing dependent apps and re-running Startup script..."
    echo "Quitting Plex Media Server..."
    osascript -e 'tell application "Plex Media Server" to quit'
    echo "Quitting Sonos..."
    osascript -e 'tell application "Sonos" to quit'
    echo "Quitting Music..."
    osascript -e 'tell application "Music" to quit'
    echo "Quitting DEVONthink..."
    osascript -e 'tell application id "DNtp" to quit'
    echo "Running Startup Apps and Databases..."
    /Applications/log-event.sh "Launching Startup Apps and Databases..."
    osascript -e 'tell application "Startup Apps and Databases" to activate'
    exit 0
fi

# If the network drive "scandocs" is not mounted, attempt re-mount
# This is only used for DEVONthink import, so no need to close the app
if [ "$scandocsMounted" = false ]; then
    echo "Network drive "scandocs" is not mounted. Attempting remount..."
    /Applications/log-event.sh "Attempting to remount network drive: scandocs..."
    osascript -e 'tell application "Finder" to mount volume "smb://mynas._smb._tcp.local/scandocs"'
fi

The LaunchAgent, com.admin.monitor-network-drives.plist should be placed in the folder ~/Library/LaunchAgents/, and then registered with this command:

launchctl bootstrap gui/<ADMIN_USER_ID> ~/Library/LaunchAgents/com.admin.monitor-network-drives.plist

Found this useful? Ran into problems? Leave a comment! Or find me on Mastodon.

1 thought on “Mac Server Reliability Scripts”

  1. Thanks for this. I don’t need all of it at the moment but I can see consulting it on every point eventually.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.