How to maintain your Emacs configuration

For Emacs users, optimizing their configuration is a fun thing, and it is also the best way to learn Emacs. Newbies are suggested to copy/paste configuration, which is the fastest and most effective way to learn. However, the configuration usually becomes more complicated and a little messy if still copy/paste after one year or two, more knowledge is required to master Emacs.

This article will introduce my experience in optimizing configuration, including mainly two part:

How Emacs load package and
Use submodule to maintain packages

Package.el issues

It is no exaggeration to say that high extensibility is the main reason why Emacs remains popular for decades. You can check how many packages you have installed through package.el with (length package-alist). It's 137 for me.

Although package.el provides a convenient way to install packages, it does not provide the function of version management. This is the most basic function of any package managers. My Emacs setup have been broken up many times because of package upgrades, which is very frustrating.

The community has some solutions, such as straight and borg, but in order to avoid introducing new problems and reduce the burden of learning, I currently do not adopt these tools. Instead, I use the submodule that comes with git to manage some heavily used packages (such as lsp-mode/magit), and do upgrade in spare time. If there is a problem with the upgrade, I can just go back to the previous commit, without having to worry about being interrupted.

How package works

For packages managed by package.el, users can use them without knowing how Emacs loads the packages, but when to manage them completely by ourselves, we need to understand these details.

First let's clarify the definition of the package:

A package is a collection of one or more ELisp files, and Emacs searches them in folders specified by load-path .

Emacs provides two high-level interfaces to automatically load packages: autoload and feature.

Autoload

1
(autoload filename docstring interactive type)

The autoload function can declare on function or macro, its corresponding file is loaded when it is used for the first time.

Generally, the autoload function is not used directly, users can write a "magic" comment in the source before the real definition. For packages installed along with Emacs, these comments do nothing on their own, but they serve as a guide for the command update-file-autoloads, which constructs calls to autoload and arranges to execute them when Emacs is built. For example, there is a hello-world.el in the my-mode folder:

1
2
3
4
;;;###autoload
(defun my-hello ()
  (interactive)
  (message "hello world"))

We can use commands below to generate its autoloads file:

1
(package-generate-autoloads "hello-world" "~/my-mode")

Then in the same directory, we get hello-world-autoloads.el

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
;;; hello-world-autoloads.el --- automatically extracted autoloads
;;
;;; Code:

(add-to-list 'load-path (directory-file-name
                      (or (file-name-directory #$) (car load-path))))


;;;### (autoloads nil "hello-world" "hello-world.el" (0 0 0 0))
;;; Generated autoloads from hello-world.el

(autoload 'my-hello "hello-world" nil t nil)

;;;***

;; Local Variables:
;; version-control: never
;; no-byte-compile: t
;; no-update-autoloads: t
;; coding: utf-8
;; End:
;;; hello-world-autoloads.el ends here

When we execute my-hello for the first time, Emacs will load hello-world.el automatically.

It should be noted here that in order for Emacs to recognize the declaration of the my-hello function, it is necessary to load the hello-world-autoloads.el first. For packages managed by package.el, when package.el downloads the package, it will perform the following operating:

Resolve dependencies, recursive download
Append the package directory to load-path
Automatically generate autoloads file and load it

In this way, users can directly use the functions provided by the package. If manage package manually, we need to implement the above operations ourselves, which will be introduced later.

Feature

Feature is another mechanism provided by Emacs to automatically load ELisp files. Take an example:

1
2
3
4
5
6
(defun my-hello ()
  (interactive)
  (message "hello world"))

;; feature should be same with filename
(provide 'hello-world)

The code above declare a feature called hello-world, in order to use my-hello we just need to (require 'hello-world), since feature is the same with filename, Emacs know which file to load.

A feature is loaded the first time when another program asks for it, subsequent requires will simply do nothing.

Load

1
(load filename &optional missing-ok nomessage nosuffix must-suffix)

Load is a relatively low-level API, although more flexible but more error-prone. Users are suggested to avoid call it directly.

Both autoloads and feature both call load to do their job.

Submodule to the rescue

When autoload is introduced above, we introduce some housekeeping when package.el downloads a package. Here is a review:

Resolve dependencies, recursive download
Append the package directory to load-path
Automatically generate autoloads file and load it

If manage package via submodule, it will only download the package itself. The above three steps need to be done by ourselves. I currently use use-package to do this. Here is an example to introduce its usage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
(use-package lsp-mode
  ;; config load-path，lsp-mode is in ~/.emacs.d/vendor/lsp-mode
  :load-path ("~/.emacs.d/vendor/lsp-mode" "~/.emacs.d/vendor/lsp-mode/clients")
  :init (setq lsp-keymap-prefix "C-c l")
  ;; config mode  hook
  :hook ((go-mode . lsp-deferred))
  ;; generate autoloads
  :commands (lsp lsp-deferred)
  ;; config custom variables
  :custom ((lsp-log-io nil))
  :config
  (require 'lsp-modeline)
  (push "[/\\\\]vendor$" lsp-file-watch-ignored-directories)
  ;; config mode-map
  :bind (:map lsp-mode-map
              ("M-." . lsp-find-definition)
              ("M-n" . lsp-find-references)))

As you can see, the use-package macro is very concise and concise, it unifies the various configurations of packages. Highly recommended. we use macroexpand-1 to see how use-package is implemented,

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
(progn
  (eval-and-compile
    (add-to-list 'load-path "~/.emacs.d/vendor/lsp-mode"))
  (eval-and-compile
    (add-to-list 'load-path "~/.emacs.d/vendor/lsp-mode/clients"))

  (let
      ((custom--inhibit-theme-enable nil))
    (unless
        (memq 'use-package custom-known-themes)
      (deftheme use-package)
      (enable-theme 'use-package)
      (setq custom-enabled-themes
            (remq 'use-package custom-enabled-themes)))
    (custom-theme-set-variables 'use-package
                                '(lsp-log-io nil nil nil "Customized with use-package lsp-mode")))
  (unless
      (fboundp 'lsp-deferred)
    (autoload #'lsp-deferred "lsp-mode" nil t))
  (unless
      (fboundp 'lsp-find-definition)
    (autoload #'lsp-find-definition "lsp-mode" nil t))
  (unless
      (fboundp 'lsp-find-references)
    (autoload #'lsp-find-references "lsp-mode" nil t))
  (unless
      (fboundp 'lsp)
    (autoload #'lsp "lsp-mode" nil t))
  (condition-case-unless-debug err
      (setq lsp-keymap-prefix "C-c l")
    (error
     (funcall use-package--warning139 :init err)))
  (eval-after-load 'lsp-mode
    '(progn
       (require 'lsp-modeline)
       (push "[/\\\\]vendor$" lsp-file-watch-ignored-directories)
       t)
    (add-hook 'go-mode-hook #'lsp-deferred)
    (bind-keys :package lsp-mode :map lsp-mode-map
      ("M-." . lsp-find-definition)
      ("M-n" . lsp-find-references))
    ))

It is almost the same as the code we manually configured.

Use-package only solve the complicated configuration problems, it does not resolve package dependencies, we need to explicitly download them one by one(we can check dependencies in Package-Requires at the head of one package):

1
2
3
4
5
6
;; lsp-mode deps
(use-package spinner
  :defer t)
(use-package lv
  :defer t)
;; ...

When use-package cannot find these dependencies in load-path, it will automatically use package.el to download them. My approach here is a compromise. For some lightweight packages, there is no need to use submodule.

Readers may think that downloading dependencies this way is too cumbersome, but in fact dependencies of different packages are likely to be the same, such as dash.el, s.el, f.el to name a few. So there are not many dependencies for manual management.

use-package bootstrap

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
(package-initialize)
(when (not package-archive-contents)
  (package-refresh-contents))

(dolist (p '(use-package))
  (when (not (package-installed-p p))
    (package-install p)))

(setq use-package-always-ensure t
      use-package-verbose t)

;; From now no, we can use use-package to config packages

Common Git commands

1
2
3
4
5
6
7
8
9
# only after sync, update .gitmodules manually will take effect
git submodule sync

# update with upstream latest commit
git submodule update --init --recursive --remote

# https://stackoverflow.com/a/18854453/2163429
# discard changes, and reset to commit specified in .gitmodules
git submodule update --init

Magit can be used for adding and removing submodule. Press o in magit-status-mode.