Keeping Your autopkg Cache So Fresh and So Clean, Clean

It’s been a minute since I felt I had something worthwhile to write about. In the past, the autopkg systems I had configured either had plenty of internal storage or external storage available to more or less “set it and forget it”. The consequence of this approach is over time if you have many software titles your autopkg cache folder will become quite large. Depending on the recipes you use, say you have many that involve naming downloaded or packaged software with the version, this can also greatly increase the size because new software version downloads and packages don’t overwrite the previous ones. So if you find yourself like I do where free space is precious on your autopkg system, you need an automated way to clean it.

While I haven’t yet taken the leap to implement autopkg in a CI workflow that makes this whole issue moot, I wrote a script to periodically clean out old downloads, packages, as well as the tmp files that can result from incomplete downloads.

As written, running this on the command line will remove any .dmg, .pkg, .tar.gz, and .zip files older than 7 days. It also removes those pesky tmp files that can get leftover from incomplete software downloads. This can also be triggered to be run as part of a LaunchDaemon.

If you wish, you can tweak the threshold for removing old files by entering your own day value for DAYS_TO_DELETE. Additionally, if you wish to exclude any items that would normally be deleted, uncomment the EXCLUSIONS variable and enter any desired search terms.

Happy cleaning!

#!/bin/bash
# Script to find and delete .dmg, .pkg, .tar.gz, and .zip files from
# your autopkg cache
# Failed / incomplete downloads (prefixed with tmp) are also removed
# Number of days after which to delete matching files
readonly DAYS_TO_DELETE=7
# Path to autopkg Cache folder for current user
readonly SEARCH_PATH="/Users/${USER}/Library/AutoPkg/Cache"
# List of search terms to exclude from cleanup, separated by spaces
#
# readonly EXCLUSIONS="Slack Firefox"
# Path to autopkg cleanup log. For CLI use, otherwise comment out
readonly LOG="/Library/Logs/autopkg_cleanup.log"
#######################################
# Output errors with timestamp to STDERR
# Globals:
# CLEANUP_LIST
# DAYS_TO_DELETE
# SEARCH_PATH
# EXCLUSIONS
# Arguments:
# None
# Outputs:
# None
#######################################
get_files_to_clean() {
CLEANUP_LIST="$(/usr/bin/find ${SEARCH_PATH} \( -iname "*.pkg" -o -iname "*.dmg" -o -iname "*.zip" -o -iname "tmp*" -o -iname "*.tar.gz" \) -mtime +${DAYS_TO_DELETE} -maxdepth 5 -print | sort)"
# Remove any excluded items from the list
for i in $EXCLUSIONS; do
CLEANUP_LIST=$(echo "$CLEANUP_LIST" | grep -v ${i})
done
}
#######################################
# Output results
# Globals:
# CLEANUP_LIST
# DAYS_TO_DELETE
# Arguments:
# None
# Outputs:
# Prints & writes to log if no files found older
# than specified DAYS_TO_DELETE, or prints & writes
# all found files to clean and indicates if it failed
#######################################
clean_files() {
# Output results
if [ -z "$CLEANUP_LIST" ]; then
# Indicate no files found to delete
writelog "No files found to clean older than ${DAYS_TO_DELETE} days."
else
# Indicate found files to delete
writelog "Files found to clean older than ${DAYS_TO_DELETE} days:"
for i in ${CLEANUP_LIST}; do
writelog "${i}"
done
# Remove found files and output any failures
for f in $CLEANUP_LIST; do
writelog "Removing $(basename ${f})"
/bin/rm -f "$f" || writelog "Failed to remove ${f}."
done
fi
}
start_run() {
writelog "####### START $(/bin/date +'%Y-%m-%d') #######"
}
end_run() {
writelog "####### END $(/bin/date +'%Y-%m-%d') #######"
}
writelog() {
# For LaunchDaemon use
#echo "$(/bin/date +'%Y-%m-%dT%H:%M:%S') – $*" #>&1
# For CLI use
echo "$(/bin/date +'%Y-%m-%dT%H:%M:%S')$*" >> "$LOG"
}
main() {
start_run
get_files_to_clean
clean_files
end_run
}
main "@"

Automatically Naming and Binding Macs in a DEP Deployment Workflow

I’ve spent the last two years refining our deployment process to make it as smooth as possible. While you may not be binding your Macs anymore, this post is meant to share our current workflow to copy or modify as you see fit. If you don’t happen to bind your Macs, that’s OK too, as this post is more focused on the automatic naming process we’ve employed which occurs prior to binding.

When we were still imaging (via Jamf Imaging & NetBoot, R.I.P.), we were naming our Macs as part of selecting the applicable Jamf Imaging config. This was prone to errors, which meant having to unbind, rename the Mac, and bind it again. Unfortunately, at the time we did not realize that the account we were using to unbind the Mac did not have sufficient privileges to remove the record in AD, so we were having to manually delete the incorrect records afterward. We subsequently implemented a workflow utilizing Jamf’s Encrypted Strings to more securely provide account passwords as part of scripts with an account that had the necessary privileges.

Below is the basic structure for this part of our large deployment workflow:

  1. A PreStage Enrollment configuration w/ DEP Macs assigned
  2. A web server with a CSV file or Google Sheet containing your Mac serial #s and associated hostnames
  3. A post-enrollment policy activated via custom trigger to automatically set our Mac’s hostname as listed in the previous file
  4. A script run at the end of the previous policy to verify the hostname and trigger either a manual rename policy (if verification fails) or another series of enrollment policies
  5. A second post-enrollment policy activated via custom trigger to bind the automatically named Mac

Giving credit where credit is due, this workflow in large part came out of the work by and discussions with an admin at Red Hook Central Schools. Here is that resulting A to Z guide for DEP enrollments with Jamf. The DEP setup & MDM configuration process is beyond the scope of this post. Additionally, a big shoutout to @haircut for creating the rename-comp.py script that makes our automated computer naming possible!

Step 1: PreStage Enrollment Configs

Coming from an imaging deployment workflow, the static or smart configuration was the basis for determining what software was installed on a given machine. With DEP, the PreStage Enrollment configurations are the new foundational layer that everything else uses to determine whether or not settings, software, configuration profiles, etc. apply for a given machine.

Having a standard naming convention for all of your PreStage Enrollments is important. Following the Red Hook Central Schools guide, it is most effective to name these based on largest to smallest grouping. Take the examples below:

Mac-Lab-MediaLab1
Mac-Lab-MusicLab1
Mac-Cart-MiddleSchool-Cart1

By starting the naming conventions with Mac-, it’s very easy to configure a smart group with a scope targeting all Macs by using the ‘PreStage Enrollment Method’ criteria which contains or starts with “Mac”.

Mac-Lab-MediaLab1
Mac-Lab-MusicLab1
Mac-Cart-MiddleSchool-Cart1

A layer deeper is the next largest grouping, which could be a mobile cart, computer lab, etc. You can target all Macs at this level by specifying PreStage Enrollment Method that contains “Mac-Lab”. Depending on your organization’s size and needs, you may only need to go one more level to define the smallest Mac grouping: the lab, cart, etc. itself. Larger organizations may benefit from specifying the school as well.

In the case of our departments, we add Dept so we can target all departments (PreStage Enrollment Method contains “Mac-Dept”) or just individual ones. And because we have separate PreStages for each department, we’re able to configure the applicable Jamf department and location information, rather than having to achieve this via other means. In our efforts to limit the overall length of these names, we’ve opted to include the division abbreviation in the smallest Mac grouping. Several examples are listed below:

Mac-Lab-USMUSLAB
Mac-Cart-MSCART
Mac-Dept-Business

Initially, it might not be immediately obvious why breaking your Mac groupings down to this granular level is worth the extra effort. In my experience it has avoided having to make major structural changes later in order to properly scope policies, software, printers, etc. However, it does mean that if you are repurposing Macs you have to be especially conscientious about what PreStage Enrollment a Mac is assigned to.

The only frustration I have with this process is the fact that Jamf does not let you reassign devices to a different PreStage Enrollment without first removing the device from its currently assigned PreStage. As a result, you either have to go to the device record or the DEP settings to access the currently assigned PreStage, remove it, and then navigate to the new PreStage and add it. There is likely room to script this process, but I also wish Jamf did not require this extra step.

Step 2: Web server with a CSV OR Google Sheet

Assuming you are using your Jamf Distribution Point for HTTP/S or munki which requires a web server to host packages, you already have the necessary mechanism in place for directing endpoints to a file with a list of serial numbers and hostnames. While there are certainly security concerns about having a single file with all your Mac serial numbers, there are well documented ways to ensure only your approved endpoints can access your local or remote web server.

While there is definitely a convenience factor using a Google Sheet for this instead, using a CSV on a web server means you can put it into a git repo and incorporate it into your change control processes.

For instructions on configuring this file – either as a CSV or a Google Sheet -, please refer to this blog post.

While the rename-comp.py script only utilizes the first two columns in the CSV or Google Sheet, we use subsequent columns to input additional notes like when Macs are decommissioned or not currently assigned to a user.

Step 3 & 4: Post Enrollment Naming Policy & Scripts

Add the rename-comp.py script to your Jamf scripts (per the previously referenced blog post). No changes need to be made to it, as the script assumes use with Jamf assigning the CSV or Google Sheet URL to parameter 4. However, you can change the default download location defined by the CSV variable, if you wish (/var/tmp/computernames.csv).

Be aware that rename-comp.py uses the Python urllib2 module, which does not exist in Python 3. As a result, changes will need to be made to this script if Apple decides to remove the now deprecated built-in Python 2.

While you can incorporate the script into a separate policy, we include this in our initial enrollment policy that all our Macs run which in turn triggers other enrollment policies. If you’re still binding your Macs, this triggering of another policy once the hostname has been changed is essential. If you include the bind setting in the same policy as your rename-comp.py script, the newly set hostname will not be used, as the hostname is only collected on initial policy run.

Hostname Verification

Because we’re human, it’s very possible that the configured CSV or Google Sheet may not contain a given machine’s serial number and/or hostname. It’s also possible that some unknown error occurs that prevents the script from grabbing your CSV file or accessing the Google Sheet. For my previous environment, this step was very important as the Mac’s hostname needed to match what was listed in AD in order to achieve Windows-like machine authentication (see this blog post for more info on that configuration).

To address these scenarios, at the very end of our initial enrollment policy we run a script which does several things:

  1. It checks that the file downloaded by rename-comp.py is where it was cached previously. If not, it redownloads it (if able).
  2. It verifies the hostname found for the Mac’s serial number in the file matches the machine’s current hostname.
  3. If the hostnames match, trigger the next enrollment policy via custom trigger (ex. jamf policy -event <trigger>) to bind and finish provisioning.
  4. If the hostname does not match – either because the file couldn’t be downloaded, the Mac’s serial number wasn’t listed, or just don’t match what’s listed in the file -, this instead triggers a different standalone policy that presents the technician with a prompt to enter the hostname manually. Once a hostname is entered, so long as the policy does not produce a non-zero exit code it triggers the next enrollment policy just like if a hostname match had occurred to finish provisioning.
#!/bin/bash
# Jamf script to be run after rename-comp.py to verify the configured hostname before continuing to provision
# a Mac.
#
# Usage:
#
# 1) Add this script to your Jamf instance
# 2) Configure the CSV variable below to match what you have listed in your rename-comp.py script
# 3) Add this script to your policy so that it runs AFTER rename-comp.py
# 4) Configure Parameter 4 for this script to match the URL configured in Parameter 4 of rename-comp.py
# 5) Configure Parameter 5 for this script to match the custom trigger configured for your policy to manually
# set the Mac's hostname
# 6) Configure Parameter 6 for this script to match the custom trigger configured for your subsequent Jamf bind
# and/or other enrollment policies
# Define path to locally cached CSV of serial #s and hostnames. This must match what's listed in
# rename-comp.py (https://gist.github.com/haircut/1debf91078ce75612bf2f0c3b3d99f03#file-rename-computer-py)
# Jamf script.
CSV='/var/tmp/computernames.csv'
# Get hostname
COMPUTERNAME=$(/bin/hostname)
# Get serial #
SERIAL=$(/usr/sbin/system_profiler SPHardwareDataType | /usr/bin/awk '/Serial Number/{print $4}')
# Script parameter 4 URL must match the parameter 4 value of the rename-comp.py
# (https://gist.github.com/haircut/1debf91078ce75612bf2f0c3b3d99f03#file-rename-computer-py) script in the
# same policy.
# Set URL to Parameter 4
URL="$4"
# Set Parameter 5 variable
MANUAL_RENAME_POLICY_TRIGGER="$5"
# Set Paramater 6 variable
ENROLLMENT_CONTINUE_TRIGGER="$6"
# Verify CSV exists, if not download it if able
if [ ! -f "$CSV" ]; then
/bin/echo "Cached CSV not found. Downloading …"
/usr/bin/curl "$URL" -o "$CSV"
fi
# Get hostname from cached CSV with serial #. Use 'head' to get first one, just in case of duplicates …
CSV_COMPUTERNAME=$(/bin/cat "$CSV" | /usr/bin/tr ',' ' ' | /usr/bin/grep "$SERIAL" | /usr/bin/head -1 | /usr/bin/awk '{print $2}')
# So long as current computer name matches what's in the CSV, continue with enrollment.
# If not, produce error, as this is likely the result of the machine not being listed in the CSV
# and switch to manual naming.
if [ "$COMPUTERNAME" = "$CSV_COMPUTERNAME" ]; then
/bin/echo "Successfully set hostname to ${COMPUTERNAME}!"
jamf policy -event "$ENROLLMENT_CONTINUE_TRIGGER"
else
/bin/echo "ERROR: Current hostname does not match or does not exist in the CSV. Please check the CSV."
/bin/echo "Switching to manual computer naming …"
# Run standalone rename policy
jamf policy -event "$MANUAL_RENAME_POLICY_TRIGGER"
# If successful, trigger enrollment continued
if [ "$(/bin/echo $?)" = 0 ]; then
jamf policy -event "$ENROLLMENT_CONTINUE_TRIGGER"
else
/bin/echo "Rename Failed. Exiting …"
exit 1
fi
fi
exit

In this way, we verify the automated hostname is what we expect before we proceed other provisioning tasks, including binding the machine. Should something go wrong, we can manually remediate without having to stop the provisioning process for the Mac and fix the list later.

Below is the script we use in our standalone rename Mac policy:

#!/bin/bash
currentCompName=$(/bin/hostname)
getCompName() {
newCompName=$(/usr/bin/osascript <<AppleScript
display dialog "Current computer name: ${currentCompName}
Please enter the new computer name below.
Example: YOUR_EXAMPLE_HOSTNAME_HERE" with title "Set New Computer Name" default answer ""
set enteredCompName to text returned of result
AppleScript
/bin/echo "${newCompName}"
)
}
verifyCompName() {
reviewNewName=$(/usr/bin/osascript <<AppleScript
display dialog "Change computer name from ${currentCompName} to ${newCompName}?" with title "Verify New Computer Name" buttons {"Cancel", "Yes"} default button 1
set response to button returned of result
AppleScript
/bin/echo "${reviewNewName}"
)
}
successNewName() {
confirmation=$(/usr/bin/osascript <<AppleScript
display dialog "Successfully changed computer name to ${newCompName}!" with title "SUCCESS" buttons {"OK"} default button 1
AppleScript
)
}
failedNewName() {
confirmation=$(/usr/bin/osascript <<AppleScript
display dialog "Failed to change computer name to ${newCompName}!" with title "FAILED" buttons {"OK"} default button 1
AppleScript
)
}
##########################
######### SCRIPT #########
##########################
getCompName
if [ "${newCompName}" = "" ]; then
/bin/echo "Rename Cancelled"
exit 0
fi
verifyCompName
if [ "${newCompName}" != "" ] && [ "${reviewNewName}" != "" ]; then
/usr/sbin/scutil –set ComputerName "${newCompName}"
/usr/sbin/scutil –set HostName "${newCompName}"
/usr/sbin/scutil –set LocalHostName "${newCompName}"
newComputerName=$(/usr/sbin/scutil –get ComputerName)
newHostName=$(/usr/sbin/scutil –get HostName)
newLocalHostName=$(/usr/sbin/scutil –get LocalHostName)
else
/bin/echo "Rename Verify Cancelled"
exit 0
fi
if [ "${newComputerName}" = "${newCompName}" ] && [ "${newHostName}" = "${newCompName}" ] && [ "${newLocalHostName}" = "${newCompName}" ]; then
/bin/echo "Rename Successful!"
successNewName
else
/bin/echo "Rename Failed."
failedNewName
exit 1
fi
exit

Step 5: Post Enrollment Bind & Other Policies

Once the Mac has been named correctly, our subsequent bind and other enrollment policies automatically run and when completed begins the software installation process.

Reflection & Looking Ahead

The addition of automated computer naming to our DEP deployment workflow has resulted in an entirely hands-off provisioning process and removed the previous potential for error when completed by hand. The only piece in the entire process that is not hands-off is setting our unattended TeamViewer Host module password and assigning it to our organization account (discussed in a previous post). TeamViewer simply doesn’t supported this (that is unless you have a Corporate license and are deploying on Windows machines).

If you wanted to go a step further with this hostname automation, you might configure your Jamf instance to trigger a webhook when a machine was added to DEP which in turn triggered another process to take the serial number from the webhook event and add it to your CSV or Google Sheet. This would avoid having to manually enter new machine serial numbers to your file. Depending on your naming scheme, you may also be able to automate the associated hostname as well. At the moment though, this process works well enough.

If you found this post helpful, give it a like. Otherwise, post your questions and comments below.

Automating Manual munki Imports with autopkg – Part 3: Pro Tools

In previous posts (macOS erase & install, Autodesk Maya), I discussed how I created recipes for importing different software titles that I’d previously processed manually.

In this third part, my focus is on Pro Tools. The challenge with automating this task is that any given release includes in the downloaded DMG the Pro Tools install as a PKG as well as both the required Codecs and HD Driver PKG installers. These supporting installs are tied to the specific Pro Tools release and which must be installed before Pro Tools. The core question then was using the available processors, could I create the necessary Pro Tools munki item while simultaneous making the applicable Codecs and HD Driver versions a part of the item’s requires array?

TL;DR – you can reassign any collected autopkg output variable to any other desired variable for later use.

See more info past the jump.

Read More