Emacs: Copy Environment Variable from Shell

Sometimes I get annoyed when my emacs session has different environment variable settings than some shell buffer I have running in the session. The most painful is when a Makefile depends on environment settings that I don't have in my .profile. Command-name completion in a shell buffer can also be painful if you have changed your path. And of course I'd like gdb to start up with the correct environment every time -- you can set the variables inside gdb, but that gets old on those days when gdb itself crashes again and again.

I kept pasting export FOO=bar into the *scratch* buffer and editing it into (setenv "FOO" "bar") and eval'ing that. After the 1000th time of doing that, I decided to automate it. Turns out emacs already has an interactive function for copying an environment variable from the shell, but you have to type in the variable name. I decided to write a little function to look for the last export, or the last echo $FOO, and copy that variable:

(defun engisneering-shell-copy-env-var ()
  (interactive)
  (let* ((expat "\\(export +\\([^=\n]+\\)=\\(.+\\)\\)")
         (echpat "\\(echo +\\$\\(.+\\)\n\\(.+\\)\\)")
         (cshpat "\\(setenv +\\([^ \n]+\\) +\\(.+\\)\\)")
         (patt (concat shell-prompt-pattern
                       "\\(" expat "\\|" echpat "\\|" cshpat "\\)")))
    (save-excursion
      (if (re-search-backward patt)
          (let* ((m (or (and (match-beginning 2) 3)
                        (and (match-beginning 5) 6) 9))
                 (var (buffer-substring (match-beginning m) (match-end m)))
                 (oldval (or (getenv var) " ")))
            (shell-copy-environment-variable var)
            (setq val (getenv var))
            (message "Old %s=%s; New %s=%s" var oldval var val))))))

Run this function inside your shell buffer, and it will search backwards for the last environment variable action and bring that variable setting into the emacs session. It's also nice because it gives you a message in the minibuffer showing the change. Add a local-set-key -- I like C-c C-v -- in your shell-mode-hook, and you're good to go.

Emacs vs. Nicer dirs

In the last post I described a nicer version of the shell dirs command that labels each directory with its position, so that you can pushd directly to the one you want.

One problem with it is that emacs shell-mode generally uses dirs to track which directory you're in. And you do want it to track directories for you. Our nicer version of dirs produces output that isn't readable to emacs.

Of course the natural solution is:

 (setq shell-dirstack-query "command dirs")

to pick up the shell's builtin dirs. Sadly, you can't just throw that line into your .emacs file, because shell-mode sets shell-dirstack-query every time you start a new shell. It sets it unconditionally and globally, which is a ridiculous design flaw, but that's how it goes. There is no hook that runs after it sets the value, either.

So you're stuck with needing a workaround:

1. Name your command something other than dirs .
2. Use a flag on your new dirs to get the nicer behavior.
3. Write your own emacs function to start a shell:

  (defun engisneering-shell ()
    (interactive)
    (shell)
    (setq shell-dirstack-query "command dirs"))

Nicer pushd, popd, and dirs

Writing up the safer, friendlier bash which command reminded me of another wrapper function that's handy to have. If you use pushd/popd to keep a stack of directories in your shell window, sometimes you want to switch to a directory far down the stack. If you want to switch to the third directory down, you do pushd +3.

The problem is, you have to count along the list of directories, since dirs, pushd, and popd all just spit out a one-line list of the stack, like this:

  $ dirs
  /usr/local/share ~ ~/work ~/tmp

Not an impossible task, but it can get annoying if you have some long paths in there, especially once your stack gets past two or three deep. Why not have dirs print one directory per line, and label them with their depths? To do so, add this function to your .bashrc file:

  function dirs {
      ds=(`command dirs`)
      i=0
      while [ "${ds[$i]}" != "" ]; do
          echo $i: ${ds[$i]};
          i=$((i+1));
      done
  }

Now you get more readable output:

  $ dirs
  0: /usr/local/share
  1: ~
  2: ~/work
  3: ~/tmp

If you want to remove ~/work from the stack, just do popd +2.

Since pushd and popd also print the directory stack, let's add the same style of output to them:

  function pushd {
      if command pushd $@ > /dev/null; then
          dirs
      fi
  }

  function popd {
      if command popd $@ > /dev/null; then
          dirs
      fi
  }

One final note. In case some wacky systemwide file has aliased these commands to something else, add the following unaliasing code to the top of your .bashrc:

  for func in dirs pushd popd; do
      if alias $func > /dev/null 2>&1 ; then unalias $func; fi
  done

Otherwise, your functions will be hidden (aliases take precedence).

PS: emacs shell-mode has a conflict with this dirs: here's how to fix it.

Bash vs. Which

The Unix which command is useful for searching your $PATH to find which version of an executable is going to run. But did you know that it doesn't always tell you the truth? For instance, it doesn't tell you when the command you're asking about is hidden by a shell builtin (or function, keyword, or alias):

  $ which echo
  /usr/bin/echo

which found /usr/bin/echo on your path, but echo is a shell builtin (at least in bash), so that's what will get run, not the one on the path. The GNU man page for which presents workarounds that will pick up functions and aliases, but shell builtins still slip through the cracks. The situation is even weirder on Solaris, where which is a csh shell-script, that sources your .cshrc file as part of its operation -- very strange if your shell is bash.

Clearly, which is not to be trusted. The easiest workaround, if you use bash, is the type builtin:

  $ type -a echo
  echo is a shell builtin
  echo is /bin/echo

A slightly nicer workaround is to make a bash function for which, that uses type to do the right thing. I also like which to do an ls -l where applicable. Here's what I have in my .bashrc:

  if alias which > /dev/null 2>&1; then unalias which; fi

  function which {
      hash -r
      t=`type -t $@`

      if [ -z "${t%%file*}" ]; then
          for p in `type -p $@`; do
              ls -l ${p};
          done
      else
          type $@;
      fi
  }

Now which gives you much more information:

  $ which -a wish
  lrwxrwxrwx  1 root  root  13 Dec 23 2008 /usr/local/bin/wish -> /usr/bin/wish*
  lrwxrwxrwx  1 root  root   7 Jan  8 2007 /usr/bin/wish -> wish8.3*

The unalias protects you in case some systemwide .bashrc or .profile aliases which in some way (unlikely, but good hygiene anyway). The call to hash clears the hashed function list, so that any changes to your PATH are reflected. The reason for the if statement in the function, is that we certainly want to report builtins, functions, etc., but if there are none, we can use type -p to make it easier to pass the filenames to ls -l. If there are builtins or such, we just punt and give the unaltered type output.

You can pass -a to this which if you want to see all the matches for the command, in order of precedence.

Note that this has to be a function, not a script, because running the script will source your .bashrc, which might change the PATH (all PATH changes should be in .profile, but sometimes people do it in .bashrc).

Csh Stupidity

One of the great things about the Linux era, is that bash seems to be taking over as the preferred shell to use, instead of csh/tcsh. The projects at my current job began during the SunOS era, so when I started there a couple years ago I went with the flow and let the sysadmin set up my default as tcsh just like everyone else there uses. When I was getting started, no one had a reasonable .profile or .bashrc I could use, so I went with tcsh so I could use the tribal .cshrc.

In terminals, the first command I run is "bash"; most of my work is in an emacs shell window, so my .emacs says (setq shell-file-name "/bin/bash") -- actually there's an (if) in there to choose cygwin's bash if I happen to be on Windows. I do a lot of shell programming at the command line, and I simply can't live with csh-style programming.

A few days ago I was working on my .cshrc file, and I noticed how dim-witted csh's if-else handling is. The classic anti-csh diatribe is Csh Programming Considered Harmful, which is an entertaining read, but there's some insanity that isn't even covered in that worthy rant. Let me start by presenting a correct csh script:

setenv FOO hello
if ($FOO == hello) then
echo hi
else if ($FOO == goodbye) then
echo see ya
endif

Source that script, and the output is:

hi

Various typos can have effects on this script that range from annoying to silent but deadly. Here's an annoying one: put the second if on a different line from the else. The output is:

hi
else: endif not found.

Well, at least you got an error message telling you something is up, although requiring two keywords to be on the same line is a level of stupidity worthy of Tcl. What if you do the opposite, and have too many endifs? Well, the output is just "hi", with no error message. I suppose that doesn't bite too much, until later when you nest another if into the script, and it suddenly gets closed by one of your stealth extra endifs.

But here's a bad one. Take the original script. Add an else at the beginning, and an endif at the end. Seems like an else without an accompanying if should get a syntax error, but it doesn't. The extra endif -- which might have been there because we always had an extra one, but never got an error about it -- closes the weird else. And guess what? The script runs with no errors, and no output. The else is kind of a "if not true". How can this be a good thing? 

[Update 2013/11/21: Ha! Now this has actually happened to me, in almost the same way described here.  I had to add some functionality to an existing script at work.  It was a branch added near the top of an if with several branches. I mistakenly added an endif at the end of my new branch. The new functionality (and the branch above it) tested fine, no error reported, so I checked it in. A few days later someone noticed that the later branches weren't running (this script gets launched from a cron job, so the problem wasn't something that would get immediately noticed). The later branches -- starting with an else for which there was no active if -- were silently skipped. Grrrr...]

I did these experiments with csh and tcsh on RedHat. Avoid csh. Long live bash!

Timestamped Shell History

Occasionally I save myself a lot of head-scratching by being able to refer to a timestamped history of my work in a particular shell session. You know the feeling, when you say either "I thought I already did that" or "Didn't I do X before Y?" Of course, usually you can piece together a timeline by looking at file modification times or revision control logs. Sometimes you absolutely cannot figure out the history -- a deleted file has no modification time -- and other times it would just be handy to have a transcript telling you when you did what.

Here are the ingredients to keeping a timestamped history:

1. Emacs shell buffer: M-x shell.
2. Timestamp script running in the background.
   
3. Lisp code to remove shell output, leaving prompts and history.

Once you get used to running your shell within emacs, you'll never go back. It keeps a complete history for you, which you can search or navigate just like any emacs buffer. History keystrokes -- M-p, M-n, and M-r as opposed to most shells' C-p, C-n, C-r -- survive past subshells, su, ssh, and even things like ftp. If you need multiple shell buffers (say, on different hosts), use M-x rename-buffer.

The hitch is that after a few weeks in your emacs session, a shell buffer can grow very large. Here is a Lisp function to put in your .emacs which knocks it back down to size (make sure the emacs shell-prompt-pattern variable matches your shell prompt):

(defun engisneering-clean-shell-buffer ()
(interactive)
(let ((beg (point-min)))
 (goto-char (point-min))
 (if (> 10000 (buffer-size)) (buffer-disable-undo))
 (while (re-search-forward shell-prompt-pattern nil t)
   (if (not (= (point) (line-end-position)))
       (progn
         (forward-line 0) ;; beginning of line, ignore prompt boundary
         (if (not (= (point) beg))
             (delete-region beg (point)))
         (forward-line 1)
         (setq beg (point)))))
 (buffer-enable-undo)
)
)

That will also be useful when you're ready to save your timestamped history. The timestamping is easy -- put this shell script into a file called "timestamp", and give it execute permission:

#!/bin/bash

while ((1)); do
   echo -n "timestamp> "
   date
   echo -n "prompt> "
   sleep 3600
done

There are two things to note about the script. First, when it outputs a timestamp, it will match the prompt pattern, so that it won't get deleted by our Lisp cleaning function. Second, it ends with "prompt> " so that the next time you hit return, emacs doesn't think the output from the timestamp is part of your next command.

So, whenever you start a shell buffer, run "timestamp&", to get the date and time printed out once an hour while you do other things.

Finally, here is a little script to help you save your shell histories. Create a directory to hold your histories; name the history files after the host the shell was on, and date them (including the year). Save them periodically, just like any of your work.

(defun engisneering-save-shell-buffer ()
(interactive)
(let ((name (buffer-name)))
 (save-buffer 0)
 (setq buffer-file-name nil)
 (rename-buffer name)
 (buffer-disable-undo)
 (delete-region (point-min) (point-max))
 (buffer-enable-undo)
)
)