Installing OpenCV via brew

Install Brew (Quickstart)

$ cd /usr/local
$ sudo ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ sudo ln -s /usr/local/homebrew/bin/brew /usr/local/bin/brew

Install opencv

$ sudo brew tap homebrew/science
$ sudo brew install -v opencv

Adjust PYTHONPATH to find opencv

$ export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH

Importing opencv

$ python
import cv

TROUBLESHOOTING

Disabling MacPorts and/or Fink

Brew’s “doctor” suggests we get rid of /sw and /opt/local from our environment if they exist, however this can prove to be cumbersome, especially when you have many different paths defined (most of which you’re not willing to lose). Below is a simple Python script that takes N paths and strips them from PATH. The resulting string is PATH excluding any matched patterns.

#!/usr/bin/env python
import os
import sys


if __name__ in '__main__':
    ARGS = sys.argv[1:]
    PATH = os.environ['PATH']
    PATH_NEW = []

    for arg in ARGS:
        for path in PATH.split(':'):
            if arg in path:
                continue
            PATH_NEW.append(path)   

    print("{0}".format(":".join(PATH_NEW)))

Example

Assume your standard PATH variable is defined as follows:

$ echo $PATH
/sw/bin:/sw/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
Purging

In order to remove the offending tree (/sw) from PATH, we execute path_purge.

$ purge_path /sw
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin

Notice that /sw is indeed missing in the output.

Putting purge_path to work

In BASH
$ export PATH=`eval purge_path /sw`
$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/texbin
In TCSH
$ setenv PATH `purge_path /sw`
$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/texbin

Using purge_path with brew

Create the following wrapper script(s):

#!/bin/bash
exclusion="/sw /opt/local"
export PATH=`eval /usr/local/bin/purge_path ${exclusion}`
export PYTHONPATH=/usr/local/lib/python2.7/site-packages
#!/bin/tcsh
set exclusion="/sw /opt/local"
setenv PATH `/usr/local/bin/purge_path ${exclusion}`
setenv PYTHONPATH /usr/local/lib/python2.7/site-packages

If you intend to run brew packages with Fink or Macports installed in parallel, issue (depending on your default shell) source ~/bin/percolate.sh or source ~/bin/percolate.csh after opening a fresh terminal.

IDL 8.2+ Segmentation Fault / Abort on OSX with Dual-Thunderbolt Displays

The Symptoms

IDL 8.2+ immediately crashes at run-time if LM_LICENSE_FILE or /Applications/exelis/license/license.dat is populated. If neither the environment variable or the license file are present, IDL enters demo mode successfully.

IDL 8.2 Crash Message

$ idl
IDL Version 8.2.3, Mac OS X (darwin x86_64 m64). (c) 2013, Exelis Visual Information Solutions, Inc.
Segmentation fault: 11

IDL 8.3 Crash Message

$ idl
IDL Version 8.3, Mac OS X (darwin x86_64 m64). (c) 2013, Exelis Visual Information Solutions, Inc.
idl(3438,0x7fff736a7310) malloc: *** error for object 0xf07: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6 

The Cause

At run-time, if licensing data is provided by the end-user, IDL attempts to gather information about the local system’s network interfaces (specifically the MAC address). In our case IDL crashes because we have two external Thunderbolt monitors connected to the computer. If we remove one of the external displays IDL is able to continue on and retrieve a license without incident.

The system experiencing the crash has eleven network interfaces defined, consisting of Bluetooth (1), FireWire (2), WiFi (1), Thunderbolt Ethernet (2), On-board Ethernet (2), Bridge Adapters (2), and an IPSec VPN (1). Whether these interfaces are active or not means nothing, as IDL will try to poll each one for a valid MAC address.

IDL 8.2 Backtrace

The data left behind in /Library/Logs/DiagnosticReports was mildly useful with IDL 8.2. The “l_getid_hw” tried to read the hardware address of a given (unknown) ethernet device, but failed miserably in the process:

VM Regions Near 0x2540be570:
    mapped file            0000000106483000-0000000107af6000 [ 22.4M] r--/r-x SM=ALI  /usr/share/icu/icudt51l.dat
--> 
    STACK GUARD            00007fff5bc00000-00007fff5f400000 [ 56.0M] ---/rwx SM=NUL  stack guard for thread 0

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libidl.8.2.dylib                    0x000000010079d6df l_getid_hw + 34 <<<< HERE
1   libidl.8.2.dylib                    0x000000010079d534 l_getid_type + 2064
2   libidl.8.2.dylib                    0x000000010079cc65 l_gethostid + 26
3   libidl.8.2.dylib                    0x00000001007866f6 checkout_from_server + 278
4   libidl.8.2.dylib                    0x000000010078400a lm_start_real + 1048
5   libidl.8.2.dylib                    0x000000010078343b l_checkout + 530
6   libidl.8.2.dylib                    0x0000000100783157 lc_checkout + 191
7   libidl.8.2.dylib                    0x00000001000ec7c2 IDL_lm_checkout2 + 184
8   libidl.8.2.dylib                    0x00000001000eddc4 d_line + 103
9   libidl.8.2.dylib                    0x00000001000eef7b init_system_routines + 699
10  libidl.8.2.dylib                    0x000000010000c306 IDL_Initialize + 3591
11  libidl.8.2.dylib                    0x000000010000c4b8 IDL_Main + 64
12  idl                                 0x0000000100000f11 main + 227
13  idl                                 0x0000000100000e0c start + 52

IDL 8.3 Backtrace

Unfortunately, IDL 8.3’s crash report is essentially useless. No relevant information was given to help trace the error.

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000

Application Specific Information:
abort() called
*** error for object 0xf07: pointer being freed was not allocated 

Conclusion

Until Exelis releases a patch to address this issue I recommend unplugging your second monitor if you need to get your work done.

Note: We did not delete the unused Thunderbolt interfaces from System Preferences -> Network to test whether this cleared things up or not. Maybe it does, maybe it doesn’t. Give it a try.

Update (09/25/2014)

Chris Torrence on the comp.lang.idl-pvwave Google group responded to my post (here):

Hi Joe,

I’ll echo what David said – excellent bug report! I believe that this bug (IDL-69092) has been fixed in IDL 8.4, due out in just a few weeks. If you’re curious, this was a bug in the FlexLM driver. Here is the bug information from Flexera:

“lmhostid instability in the presence of multiple ethernet adapters. Previously, in the presence of several ethernet adapters, lmhostid and the callers of the lc_hostid API may experience a segmentation fault on OS X. This is now resolved. (FNP-9885)”

Apparently, Matlab and other products ran into the same issue.

Joe, once you’ve upgraded to IDL 8.4, could you post here and let us know if it’s fixed? If it’s still an issue, definitely contact support@exelisvis.com.

Cheers,

Chris

ExelisVIS

(Thanks again Chris!)

Update (10/24/2014)

Great news!

Now that we finally got our hands on IDL 8.4 I wanted to confirm this update did fix the segmentation fault issue on the machines in question.

Ureka, Eclipse+PyDev Quickstart

The Eclipse interface differs slightly between operating systems (particularly OSX). If you are regular user of Eclipse, you should not have much trouble following along. Otherwise, I suggest flipping through the Eclispe documentation before diving in.

Install Eclipse

http://wiki.eclipse.org/Eclipse/Installation

Install PyDev

  1. Click Help
  2. Select Eclipse Marketplace...
  3. Under the Search tab select Yaxos Marketplace (the orange icon)
  4. In the Find: textbox enter “pydev”, and click Go or press the [Enter] key
  5. Select “PyDev for Eclipse”
  6. Click Install
  7. Proceed through the plugin installation dialog(s).
  8. Restart Eclipse when prompted
  9. Select the PyDev perspective (upper-right-hand corner of the Eclipse interface)

PyDev Interpreter Setup

Important note: Step 7 grants you the ability to import modules installed via pip within the Ureka environment.

  1. Click Window
  2. Select Preferences
  3. Expand the PyDev settings tree
  4. Select Python Interpreter
  5. Click New...
    • Interpreter Name: Ureka
    • Interpreter Executable: /path/to/Ureka/python/bin/python2.7
  6. Click Ok
  7. With the Ureka interpreter highlighted, click New Folder
    • Location: /path/to/Ureka/variants/common/lib/python2.7/site-packages
  8. Click Ok to close the New Folder dialog box
  9. Click Ok to apply the new interpreter settings

PyDev Editor Settings (Personal Touch)

I seriously recommend enabling line trimming if you intend to share your code with others.

  1. Repeat steps 1, 2, and 3
  2. Expand the Editor settings tree
  3. Expand the Code Style settings tree
  4. Select Code Formatter ( ^ = CHECKED, _ = UNCHECKED )
    • [^] Auto-format only files in the workspace?
    • [_] Use autopep8.py for code formatting?
    • [_] On save, only apply formatting in changed lines?
    • [^] Use spaces after commas?
    • [_] Use space before and after parenthesis?
    • [_] Use space before and after assign for keyword arguments?
    • [^] Use space before and after operators? (+, -, /, *, //, **, etc.)
    • [^] Right trim lines?
    • [^] Right trim multi-line string literals?
    • [^] Add new line at end of file?
    • [2 spaces] Spaces before comment?
    • [At least 1 space] Spaces in comment start?

Optimizing Ureka

As promised in my previous post, I have decided to release the Ureka optimization script I’ve been working on. Head over to the ur_optimize project site for more details: click here

Enjoy!


Why do I want to use ur_optimize?

The version of ATLAS bundled with Ureka (http://ssb.stsci.edu/ureka) is designed to work on a large number of CPUs, therefore certain operations requiring linear algebra may perform inadequately to your needs.

ur_optimize rebuilds the underlying ATLAS/LaPACK/BLAS stack to be tailored for your CPU’s architecture. After that, the NumPy/SciPy stack is recompiled and linked against the new ATLAS implementation in order to take advantage of these optimizations.

License

BSD

Requirements

RHEL 6.0+ / Fedora 10+

sudo yum install gcc gfortran

Ubuntu 12.04+

sudo apt-get install build-essential

Testing ur_optimize

For this test I used the latest Ureka build (as of 09/04/2014):

Ureka_linux-rhe5_64_1.4.1.tar.gz

SYSTEM

Manufacturer: Dell Inc.
Product Name: OptiPlex 790

BIOS

Vendor: Dell Inc.
Version: A07
Release Date: 09/10/2011

CPU

vendor_id   : GenuineIntel
cpu family  : 6
model       : 42
model name  : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping    : 7
microcode   : 0x29
cpu MHz     : 3399.867
cache size  : 8192 KB
physical id : 0
siblings    : 8
core id     : 3
cpu cores   : 4

MEMORY INFORMATION

size            : 16GB
type            : DDR3
Mhz             : 1333

test_linalg.py module

#!/usr/bin/env python
import numpy
import scipy.interpolate

def interpolate(user_methods=None, points=1000):
    x = numpy.linspace(1.0, 1000.0, num=points)
    y = numpy.linspace(1.0, 1000.0, num=points)

    methods = ['linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic']
    if user_methods is not None:
        methods = user_methods

    for method in methods:
        print("{}...".format(method)),
        try:
            scipy.interpolate.interp1d(x, y, kind=method, bounds_error=False, fill_value=0.0)
        except MemoryError:
            # Common ocurrance with quadratic
            print("failed")
        
        print("ok")

if __name__ == "__main__":
    interpolate()

Performance Comparison

Before ur_optimize

$ ipython
Python 2.7.5 (default, Jun 19 2014, 11:22:38) 
Type "copyright", "credits" or "license" for more information.

IPython 2.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import test_linalg

In [2]: %timeit test_linalg.interpolate(['quadratic'])
quadratic... ok
quadratic... ok
quadratic... ok
quadratic... ok
1 loops, best of 3: 39 s per loop

39 seconds per operation.

After ur_optimize

$ ipython
Python 2.7.5 (default, Jun 19 2014, 11:22:38) 
Type "copyright", "credits" or "license" for more information.

IPython 2.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import test_linalg

In [2]: %timeit test_linalg.interpolate(['quadratic'])
quadratic... ok
quadratic... ok
quadratic... ok
quadratic... ok
1 loops, best of 3: 1.17 s per loop

1.17 seconds per operation.

Found a bug?

If ATLAS fails to compile for your architecture, please contact the developers of ATLAS, not me: http://math-atlas.sourceforge.net/faq.html#help

Otherwise, feel free to submit a bug report on this project’s issue tracker: http://bitbucket.org/jhunkeler/ur_optimize/issues

Eureka! Erm… Ureka!

What is Ureka?

Ureka serves as an all-purpose science solution, that allows users to test different python modules and environment configurations separate from their standard shell environment. The software distribution also includes many common science tools, such as, IRAF 2.16, SAO DS9, S(ource)Extractor, Python (astropy, numpy, scipy, etc). Ureka’s subset of GNU libraries is quite extensive for a relatively small package (~3GB). A bulk of its total size is IRAF, which is an unavoidably huge science package, but Ureka has outdone itself by taming the beast into a self-contained structure (unlike installing it from scratch), so you can spend more time focusing on functional work, rather than troubleshooting missing trailing slashes.

Ureka’s core relies heavily on virtualenv and makes creating “variants” of your current installation extremely easy. As a developer, if you require ten different Python installations this distribution should make your life much easier.

Installation

Refer to this page for more detailed installation information.

The shortest quickstart guide in the history of science software, ever:

$ wget http://ssb.stsci.edu/ureka/1.4/install_ureka_1.4
$ chmod +x install_ureka_1.4
$ ./install_ureka_1.4
...
$ source ~/.bashrc
$ ur_setup

Performance

Compared to other software distributions in the same league, such as Anaconda, Ureka is on-par. The added benefit of Ureka’s built-in complement of science tools makes everything a bit more palatable. Would you rather compile IRAF and link it against Anaconda’s libraries all by your lonesome? How about SAODS9? I seriously doubt it. Ureka has the upper hand in this regard but there is one area where performance falls unavoidably short: ATLAS (Automatically Tuned Linear Algebra Software).

Anyone I have ever spoken with about ATLAS, even mentioning it in passing, starts out the same way: The person involuntarily rolls their eyes and lets out a huge painful sigh as if they are reliving a recurring nightmare from their childhood. Every single person complains it was one of the most annoying pieces of software to compile, ever.

ATLAS is “automatically tuned” (i.e. it benchmarks itself against your physical system hardware at compile-time) and therefore makes releasing an even marginally functional pre-compiled shared library impossible.

Ureka’s linear algebra performance on Linux is not very impressive. All hope is not lost! If you are willing to give up an afternoon… Manually compiling ATLAS+LAPACK(Tuned) then re-linking SciPy and NumPy via pip works fantastically. I’ve written a script to do this automatically, so when I’ve ironed out all the kinks I’ll post a how-to article.

The Science Software Branch at STScI has gone through a lot of effort to make this distribution easy to use on a day-to-day basis. So even if you’re not receiving the best number crunching capabilities out of the box, you’re already making up lost time not building/installing extremely difficult to understand, often poorly documented and barely maintained scientific tools that everybody relies on.

Note

ATLAS performance degradation is limited to Linux only.
OSX provides VecLib (Apple’s in-house build of ATLAS/BLAS/LAPACK) which Ureka’s linear algebra dependent software is linked against by default. So if you own an iThing you’re OK for the time being.

Potential Caveats

  • Upgrading the Ureka distribution destroys changes made by the user.
  • Wrapping of system-provided utilities could lead to unwanted effects.

I understand the need for some of these hacks from a development standpoint… I am not sure I agree with them being implemented in a production build.

Environment Variables

IRAF’s nightmare of a FORTRAN compiler wrapper is unfortunately here to stay, so if you decide to compile code while Ureka is active, due to library linkage issues you may need to redefine the following variables (i.e., F77=gfortran, F2C=gfortran) for normal compilation to work again.

$ which $F77
/opt/Ureka/iraf/unix/hlib/f77.sh

$ which $F2C
/opt/Ureka/iraf/unix/bin.linux/f2c.e

GCC

The wrapping of GCC can be undesirable if you rely on a tool-chain other than the system’s default.

$ which `gcc`
/opt/Ureka/python/bin/gcc

The following error is benign but may be confusing to a newcomer:

$ gcc
Undefined symbols for architecture x86_64: "_main", referenced from: 
    implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Because gcc returns 1 instead of 0, this behavior may interfere with scripts that check for the existence and/or functionality of GCC.

PKG-CONFIG

If you rely on pkg-config (most GNU software does) consider removing this wrapper script, or be prepared to set -L and -I arguments manually in CFLAGS/LDFLAGS, etc.

$ which pkg-config
/opt/Ureka/python/bin/pkg-config

$ cat `which pkg-config`
#!/bin/sh

case "$*"
in
    *freetype2*|*cfitsio*)
        /usr/bin/pkg-config $*
        ;;
    *)
        exit 126
        ;;
esac

OTOOL (OSX)

If otool doesn’t exist on your Apple iThing there is a substantially high probability Ureka isn’t the only piece of software no longer able to check library linkage. However, it does appear to be supplied within the distribution.

$ which otool
/opt/Ureka/python/bin/otool

$ diff /usr/bin/otool `which otool`
Binary files /usr/bin/otool and /opt/Ureka/bin/otool differ

INSTALL_NAME_TOOL (OSX)

This Apple iThing system utility for changing RPATHS is included in the distribution as well.

$ which install_name_tool
/opt/Ureka/bin/install_name_tool

$ diff /usr/bin/install_name_tool `which install_name_tool`
Binary files /usr/bin/install_name_tool and /opt/Ureka/install_name_tool differ

Operation not permitted: Sticky and SETGID

  1. User “foo” creates a directory with group read-write and sticky, without setgid
    [foo@localhost ~]$ mkdir -p /tmp/test_sticky/test_setgid/{sub1,sub2,sub3}
    [foo@localhost ~]$ chmod 3775 /tmp/test_sticky
    
  2. User “foo” then creates a group read-write, setgid, non-sticky directory structure with few files underneath.
    [foo@localhost ~]$ chmod 2775 /tmp/test_sticky/test_setgid
    [foo@localhost ~]$ find /tmp/test_sticky/test_setgid -type d -exec chmod 775 "{}" \;
    [foo@localhost ~]$ touch /tmp/test_sticky/test_setgid/{sub1/file1,sub2/file2,sub3/file3}
    
  3. The resulting structure looks like this:
    [foo@localhost ~]$ find /tmp/test_sticky -printf "%#m:%M:%u:%g:%p\n"|sort -n
    0644:-rw-r--r--:foo:shared:/tmp/test_sticky/test_setgid/sub1/file1
    0644:-rw-r--r--:foo:shared:/tmp/test_sticky/test_setgid/sub2/file2
    0644:-rw-r--r--:foo:shared:/tmp/test_sticky/test_setgid/sub3/file3
    02775:drwxrwsr-x:foo:shared:/tmp/test_sticky/test_setgid
    02775:drwxrwsr-x:foo:shared:/tmp/test_sticky/test_setgid/sub1
    02775:drwxrwsr-x:foo:shared:/tmp/test_sticky/test_setgid/sub2
    02775:drwxrwsr-x:foo:shared:/tmp/test_sticky/test_setgid/sub3
    03775:drwxrwsr-t:foo:shared:/tmp/test_sticky
    
  4. User “bar” attempts to remove /tmp/test_sticky/test_setgid:

    [bar@localhost ~]$ rm -rfv /tmp/test_sticky/test_setgid
    removed `/tmp/test_sticky/test_setgid/sub3/file3`
    removed directory: `/tmp/test_sticky/test_setgid/sub3`
    removed `/tmp/test_sticky/test_setgid/sub2/file2`
    removed directory: `/tmp/test_sticky/test_setgid/sub2`
    removed `/tmp/test_sticky/test_setgid/sub1/file1&`
    removed directory: `/tmp/test_sticky/test_setgid/sub1`
    rm: cannot remove `/tmp/test_sticky/test_setgid`: Operation not permitted
    

    The sticky bit set by “foo” on /tmp/test_sticky prevented “bar” from deleting
    /tmp/test_sticky/test_setgid, effectively overriding the setgid permissions.

  5. Deleting the test_setgid directory as “bar” without the sticky bit enabled
    [ As “foo”, drop the permissions back to setgid only: chmod 02775 /tmp/test_sticky ]

    [bar@localhost ~]$ rm -rfv /tmp/test_sticky/test_setgid
    removed directory: `/tmp/test_sticky/test_setgid`
    

Launch Aquamacs from within a shell

Problem

  • Opening Aquamacs from the shell (/Applications/Aquamacs.app/Contents/MacOS/Aquamacs) throws many deprecation warnings.
  • Multiple instances of Aquamacs are appended to the dock when launched, and it is annoying.

Solution

  1. Open Aquamacs.app
  2. Click “Tools
  3. Click “Install Command Line Tools

Resolution

The path to Aquamacs after installing the command line tool package is /usr/bin/aquamacs

Building CATAPACK on OS X

What?

CATAPACK is a package for management and manipulation of photometric catalogues of stellar fields, particularly suitable for the determination of accurate astrometric solutions […] (ref)

Download

I could not find a copy of CATAPACK on the internet, however the source code is GPLv2, so I have decided to provide the software on this site.

Link: CataPack-2.1.19.tar.gz

Update (06/19/2014)

CataPack found itself a maintainer and a minor revision bump as well…

Main site: CataPack
Link: CataPack-2.2.4.tar.gz

Building

Prerequisites

Dependencies

Compiling

Install dependencies

sudo port install gsl gtk-engines wcslib wcstools

Unpack the source

tar xf CataPack-2.1.19.tar.gz
cd CataPack-2.1.19

Remove the usage of ancient RSML entries

/usr/bin/sed -i '' \ 
-e 's|.so|#.so|' \ 
src/CataCal \ 
src/CataCal.in \ 
src/gsc2 \ 
src/gsc2.in

Build the source

./configure --prefix=/usr/local
make

Install CATAPACK

sudo make install

 

Remote filesystems and mlocate

Have you ever worked in a clustered environment that provided remote home directories?  Furthermore, have you ever been annoyed by the fact mlocate does not descend into these remote file systems by default?  Me too.  I’ve written two small BASH scripts to address this problem.

updatedb_extern

#!/bin/bash

# Edit DEST to point to a local directory you have write access to
DEST=/path/to/local/database/directory

#Don't edit below this line (no point)
EXTERN=( "$@" )
DATABASES=()

#Did we receive any paths to process?
if [ -z "$EXTERN" ] ; then
	echo "No path(s) specified."
	exit 1
fi

#Simple adapation logic to ensure we will have a writable data area
if [ ! -d "$DEST" ] ; then
	mkdir -p $DEST 2>/dev/null
	if [ $? != 0 ] ; then
		DEST=/var/tmp/mlocate
		mkdir -p $DEST 2>/dev/null
		if [ $? != 0 ] ; then
			echo "No suitable path to store locate database."
			exit $?
		fi
	fi
fi

#Generate database names based on external path
#Example: /home/myuser becomes _home_myuser.db
for path in "${EXTERN[@]}"
do
	database=$(echo $path | sed -e "s|/|_|g")
	DATABASES[${#DATABASES[*]}]="$DEST/${database}.db"
done

#For each external path generate an mlocate database
MAX=${#EXTERN[@]}
for (( i=0; i<$MAX; i++))
do
	i_fake=$(( i + 1 ))
	path=${EXTERN[$i]}
	dbpath=${DATABASES[$i]}
	echo "[$(echo $(( $i_fake * 100 / $MAX )))%] $path -> $dbpath"
	updatedb -l 0 -o $dbpath -U $path
done
echo

The developers of mlocate were nice enough to implement the environment variable, LOCATE_PATH, to extend the database search path.  We’re going to use it to our advantage with the following script.

updatedb_extern_setup

#!/bin/bash

DEST=$1
if [ -z "$DEST" ] ; then
	echo "No database path specified."
	exit 1
fi

for db in $DEST/*.db
do
	DELIM=":"
	if [ -z "$LOCATE_PATH" ] ; then
		DELIM=
	fi

	LOCATE_PATH=${LOCATE_PATH}${DELIM}${db}
done
echo "$LOCATE_PATH"

 

Example Usage

In your ~/.bashrc or ~/.bash_profile (or ~/.profile):

export LOCATE_PATH=`update_extern_setup /path/to/local/database/directory`

In your crontab (e.g. crontab -e):

* 2 * * * updatedb_extern /home/username /some/remote/path

Manually:
updatedb_extern /home/username /some/remote/path