Emacsclient and TRAMP

For quite a while now, I’ve done a large amount of my text editing in emacs over ssh using TRAMP. It’s extremely convenient to be able to run one instance of emacs locally instead of spawning an editor remotely and interacting with it over ssh. I’ve tried using sshfs in the past, but it’s never worked well for me in the general case. But, this isn’t a post about justifying my choices.

The other crucial part of my local emacs setup has been emacsclient. Setting my local $EDITOR to emacsclient means that I could, for example, run git commit in a terminal and edit the commit message without having to spawn a new emacs instance. Instead, emacsclient tells the existing emacs what files to edit, and waits until emacs says it’s done with them.

Until just yesterday, there was a major flaw in my setup: I couldn’t activate my local emacs instance through emacsclient on a remote host. This meant that $EDITOR had to be set to something that spawned an editor on the remote host, which was becoming increasingly frustrating.

In the past I’d tried doing things like using an ssh tunnel to forward the emacsclient connections from the remote host to my local machine, but emacsclient doesn’t anticipate this case. The emacsclient protocol is very simple: ASCII commands delimited with line feeds. So, for example:

1
2
3
4

$ emacsclient /foo/bar
# and this sends (approximately):
-auth some-gunk-here
-dir /foo -file /foo/bar

The problem is that this is being sent to your local emacs, which then tries to load the local /foo/bar. A TRAMP path is in the form of (approximately) /ssh:example.com:/foo/bar, so if only this could be prepended, emacs could deal with it!

In my most recent investigation of the problem, I found Ryan Barrett’s implementation, which was most of what I wanted. Except,

  1. It wasn’t automatic. It’s really convenient to be able to have TRAMP dump the appropriate authentication file onto the server as soon as a connection is made without having to connect using a special function.

  2. It was relying on emacsclient sending a -tty argument in a specific format, and then using the TRAMP prefix of the current buffer. In practice this means you have to have your frontmost buffer a file open on the remote server, and only then can you use emacsclient on that remote server.

I started to hack on it, but I was still affected by the same original problem: emacsclient doesn’t know about TRAMP. The default implementation doesn’t even have a way of specifying additional data to be sent over the wire. With the simplicity of the emacsclient protocol, it started looking like writing my own would be the simplest choice.

So, I put together some elisp to write out a special client authentication file that would also include the TRAMP prefix and a shell script that would parse it and write commands using nc. My elisp implementation borrows code from Ryan Barrett’s (and thank you for getting me started with this!) but goes a bit farther. The full workflow is like this:

  1. Update your .emacs to load and configure the elisp.

  2. Update your .profile to set $EDITOR to the emacsclient.sh script and tell it where the authentication file is.

  3. Open a TRAMP connection from your local emacs to write out the authentication file.

  4. Go hog wild.

  5. Repeat only steps 3-5.

Here are the versions of emacsclient.sh and remote-emacsclient.el as of 2013-06-25:

(emacsclient.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

#!/bin/sh
nowait=0
if [ "$1" = "-n" ]; then
    nowait=1
    shift
fi

if [ "$#" -eq 0 ]; then
    echo "no files specified"
    exit 0
fi

[ -e "$remote_emacs_auth" ] && client_host=$(sed -n 1p "$remote_emacs_auth")
if [ -z "$client_host" ] || ! nc -z $client_host; then
    echo "no emacs server"
    sleep 1
    exec emacs "$@"
fi

quote () {
    sed 's:&:\&\&:g;s:-:\&-:g;s: :\&_:g'
}

quoteline () {
    printf "%s\n" "$1" | quote
}

unquote () {
    sed 's:&&:\&:g;s:&-:-:g;s:&_: :g;s:&n:\
:g'
}

client_auth=$(sed -n 2p "${remote_emacs_auth}")
tramp_prefix=$(sed -n 3p "${remote_emacs_auth}")
args=$(printf "%s\n%s\n" "${client_auth}" "-dir ${tramp_prefix}${quoted_pwd}")
[ "${nowait}" != 0 ] && args="${args} -nowait"
quoted_pwd=$(quoteline "$(pwd)")

for file; do
    quoted_file=$(quoteline "${file}")
    case "${file}" in
        +*) argument="-position ${quoted_file}";;
        /*) argument="-file ${tramp_prefix}${quoted_file}";;
        *)  argument="-file ${tramp_prefix}${quoted_pwd}/${quoted_file}";;
    esac
    args="${args} ${argument}"
done

printf "%s\n" "${args}" | nc $client_host | unquote
(remote-emacsclient.el) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123

(require 'tramp)
(require 'tramp-sh)

(setq server-use-tcp t)
(server-start)

(defun put-alist (key value alist)
  "Set cdr of an element (KEY . ...) in ALIST to VALUE and return ALIST.
If there is no such element, create a new pair (KEY . VALUE) and
return a new alist whose car is the new pair and cdr is ALIST."
  (let ((elm (assoc key alist)))
    (if elm
        (progn
          (setcdr elm value)
          alist)
      (cons (cons key value) alist))))

(defun update-tramp-emacs-server-port-forward (method-name)
  "Update the specified TRAMP's method to forward the Emacs
 server port to the local host. This lets emacsclient on the
 remote host open files in the local Emacs server."
  (let* ((method (assoc method-name tramp-methods))
         (ssh-args (cadr (assoc 'tramp-login-args method))))
    (put-alist 'tramp-login-args
      (list (put-alist "-R" (let ((port
                                   (process-contact server-process :service)))
        ;; put-alist makes a dotted pair for the key/value, but tramp-methods
        ;; needs a normal list, so put the value inside a list so that the
        ;; second part of the dotted pair (ie the cdr) is a list, which
        ;; converts it from a dotted pair into a normal list.
                              (list (format "%s:127.0.0.1:%s" port port)))
                       ssh-args))
      method)))

(defun tramp-make-tramp-file-name-from-vec (vec file)
  "Convenience function for making a TRAMP path, since this
apparently didn't already exist."
  (tramp-make-tramp-file-name
    (tramp-file-name-method vec)
    (tramp-file-name-user vec)
    (tramp-file-name-host vec)
    file))

(defcustom tramp-default-remote-emacsclient-auth-file
  "~/.emacs.d/remote-server"
  "Default remote path at which to save the remote emacsclient
authentication file. This can be a string or nil to disable
saving an authentication file.

The authentication file is similar to the one written out by the
emacsclient TCP server, except it includes the prefix used for
the TRAMP connection to the remote server."
  :group 'tramp
  :type '(choice (const nil) string))

(defcustom tramp-remote-emacsclient-auth-file-alist nil
  "The remote emacsclient authentication file path to use for
specific host/user pairs. This is an alist of items (HOST USER
PATH). The first matching item specifies the path to use for a
connection which does not specify a method. HOST and USER are
regular expressions or nil, which is interpreted as a regular
expression which always matches. If no entry matches, the
variable `tramp-default-remote-emacsclient-auth-file' takes
effect.

If the connection does not specify the user, lookup is done using
the empty string for the user name.

See `tramp-default-remote-emacsclient-auth-file' for an
explanation of the auth file path."
  :group 'tramp
  :type '(repeat (list (choice :tag "Host regexp" regexp (const nil))
             (choice :tag "User regexp" regexp (const nil))
             (choice :tag "emacsclient auth path" string (const nil)))))

(defun tramp-get-remote-emacsclient-auth-file (vec)
  "Determine the full TRAMP path for the remote emacsclient
authentication file, given a connection vector."
  (let
      ((auth-file
        (let ((choices tramp-remote-emacsclient-auth-file-alist)
              (host (or (tramp-file-name-host vec) ""))
              (user (or (tramp-file-name-user vec) ""))
              lfile item matched)
          (while choices
            (setq item (pop choices))
            (when (and (string-match (or (nth 0 item) "") host)
                       (string-match (or (nth 1 item) "") user))
              (setq lfile (nth 2 item)
                    choices nil
                    matched t)))
          (if matched lfile tramp-default-remote-emacsclient-auth-file))))
    (if auth-file
        (tramp-make-tramp-file-name-from-vec vec auth-file))))

(defun tramp-save-remote-emacsclient-auth-file (&optional vec)
  "Write the remote emacsclient authentication file for a given
connection buffer, or, if used interactively, for the TRAMP
connection of the current buffer."
  (interactive)
  (let ((vec (or vec (tramp-dissect-file-name default-directory))))
    (condition-case err
        (let ((auth-file (tramp-get-remote-emacsclient-auth-file vec))
              (server (process-contact server-process :local)))
          (if auth-file
              (with-temp-file auth-file
                (insert
                 (format "127.0.0.1 %d\n" (elt server (- (length server) 1)))
                 (format "-auth %s\n" (process-get server-process :auth-key))
                 (server-quote-arg (tramp-make-tramp-file-name-from-vec vec ""))
                 "\n"))
            (when (called-interactively-p 'any)
              (message "No remote emacsclient auth file for %s"
                       default-directory))))
      (file-error (message "error saving remote emacsclient auth: %s" err)))))

(defadvice tramp-open-connection-setup-interactive-shell
  (after copy-server-file-by-tramp (proc vec) activate)
  "Automatically write out a remote emacsclient auth file after a
successful connection."
  (tramp-save-remote-emacsclient-auth-file vec))

(provide 'remote-emacsclient)

The most up-to-date versions of each will be in my dotfiles repository. For convenience, here are links to the files within the repository: emacsclient.sh and remote-emacsclient.el.

Configuration is pretty straightforward. In my local .emacs file, (server-start) got changed to:

1
2

(require 'remote-emacsclient)
(update-tramp-emacs-server-port-forward tramp-default-method)

The update-tramp-emacs-server-port-forward function takes a TRAMP method and updates the ssh arguments to that method to include a -R flag forwarding remote TCP connections to the local server. It’s probably not necessary to call it for any TRAMP method other than your default.

By default every new server connection will leave an authentication file. This can be tuned with the tramp-default-remote-emacsclient-auth-file and tramp-remote-emacsclient-auth-file-alist variables. The former can be set to nil to disable auth file creation by default. The latter is in the same form as the tramp-default-method-alist variable for specifying the location of the auth file by host and/or by user.

And in my .zshrc:

1
2
3

: ${remote_emacs_auth:="$HOME/.emacs.d/remote-server"}; export remote_emacs_auth
export EDITOR="$HOME/.dotfiles/emacsclient.sh"
alias emacs="$HOME/.dotfiles/emacsclient.sh -n"

$remote_emacs_auth’s default mirrors remote-emacsclient.el’s default of writing the auth file to ~/.emacs.d/remote-server. Since emacsclient.sh is (for me, by default) checked out in ~/.dotfiles and set executable, this is all that’s required to use it as an editor. As an added bonus, the emacs alias lets me type emacs somefile over ssh to open somefile in my local emacs.

There’s only two caveats (that I’m aware of (I’ve only tried the scp, ssh, and scpc tramp methods with this so far)):

  1. Since the authentication key changes every time emacs is started, you must open a new tramp connection to a server before emacsclient will be able to speak to your local emacs. I haven’t found a good solution to this other than forcing the authentication key to always be the same (bad) or making emacs reconnect to everything whenever you start it (worse).

  2. There’s no detection of when the server is restarted. Restarting the server in a running emacs also changes the authentication key. Making a new tramp connection isn’t required in this case; there’s a tramp-save-remote-emacsclient-auth-file function that can be run interactively. Running it will write a new auth file for the connection corresponding to the current buffer.

TRAMP, emacs
Comments

Comments