References/Debian (cont.d)

A ThinkPad X41 was turned into OPNsense appliance.

Why OPNsense? Both pfSense and OPNsense are derivatives of the original m0n0wall. But according to Wikipedia/OPNsense, pfSense created a website to abuse its rival OPNsense. Maybe this is the reason why m0n0wall put up a banner on its website.

“… and I encourage all current m0n0wall users to check out OPNsense”

— Manuel Kasper (author of m0n0wall)

Miscellaneous notes

[TODO]: perl syntax on boost.org

python 3 syntax on python.org

• ., match any single character
• ?, match optional preceding single character
• (), together with ?, specifying the range, match optional preceding token within the parentheses
• |, logic OR
• [], match any single character listed within the brackets

• +, match preceding token 1 or more times

  Example: (ht|f)tps?://.+, match http, https, ftp URLS

• ^, match the beginning letter of line

• $, match the ending letter of line

  Example: ^.+(jpg|gif|png)$, match jpg, gif, png files

  Example: C.+?c, match between C and first c we find; ? makes matching occur once

  Example: Adding quotation marks at the beginning and end of each line of a file within emacs. M-x replace-regexp RET ^ RET " RET will add double quotation mark at the beginning of each line; similarly M-x replace-regexp RET $ RET " RET will add double quotation mark at the end of each line. To remove quotation marks at the beginning and end of each line, simply replace every " with a blank.

• Cinelerra has a render farm feature which can be deployed either on slave PCs within the same network as the master PC or on the master PC itself with duplicates of itself as slave PCs. The latter scenario works since the TCP communications happen between multiple localhost, i.e., the master PC itself.

  

  In the above example, six concurrent rendering jobs are to be created. To start six cin -d PORT daemons with specified ports as above, use a for loop in a bash console,

  $for i in {24..29}; do cin -d 540$i ; done
  

  To concatenate resulting video pieces, see FFmpeg Concatenate.

• Below is my bash script to batch convert videos to *.webm according to Google’s recommended settings. The handling of passing script options may not be optimal. Any comments are very welcome.

  #!/usr/bin/env bash
  set -e          # err exit
  #set -u         # treat undefined as err
  set -o pipefail # if one pipe fails, entire chain fails
  
  if [[ $# -lt 1 ]]; then
      echo "type $(basename $0) -h for usage"
      exit 1
  fi
  
  # passing script options 
  # -r: resolution, source video resolution height. 1080+ > 1080 > 720 > 480 > 360 > 360-
  # -a: all, convert with specified res and all res below specified
  
  while getopts ":r:a:h" OPTIONS; # leading : means invalid options default to ? case
  do
      case $OPTIONS in
          r)
              RESOLUTION=$OPTARG
              echo "choose resolution $RESOLUTION"
              ;;
          a)
              ALL=$OPTARG            
              echo "choose -a setting $ALL"
              ;;
          h)
              echo "Usage: $(basename $0) [-r resolution] [-a y/n] files" >&2
              echo "-r : Video height.                                  "
              echo "     This height lies in one of catagories below,   "
              echo "     1080+ > 1080 > 720 > 480 > 360 > 360-          "
              echo "-a : Convert with the specified resolution only or  "
              echo "     with all resolutions below. Use y(es) or n(o). "
              exit 0
              ;;
          \?)
              echo "type $(basename $0) -h for usage"        
              exit 1
              ;;
      esac
  done
  shift "$(($OPTIND -1))"
  
  # workaround for absent -r -a
  # getopts does not support optional arguments
  
  # sanity check resolution
  if [[ $RESOLUTION == "" ]]; then
      echo "Need to specify target resolution"
      echo "type $(basename $0) -h for usage"        
  
      exit 1
  else
      re='^[0-9]+$' 
      if ! [[ $RESOLUTION =~ $re ]] ; then
          echo "Resolution $RESOLUTION is not an integer" >&2; exit 1
      fi
  fi
  # sanity check -a setting
  if [[ $ALL != "y" ]] && [[ $ALL != "n" ]]; then
      echo "-a option can only be 'y' (for yes) or 'n' (for no)!"
      exit 1
  fi
  # sanity check file list
  FILES=$*
  if [[ $FILES == "" ]]; then
      echo "Need to name the files to be processed"
      echo "type $(basename $0) -h for usage"        
  
      exit 1
  fi
  
  # passed all above; show what to be processed
  echo "Will process following files: $FILES"
  
  # how to write script which takes input arguments
  # https://www.lifewire.com/pass-arguments-to-bash-script-2200571
  
  # specify the file name here
  for FILE in $FILES # "$@" # an array of inputs
  do
  
      if [ ! -f $FILE ]; then
          echo "$FILE does not exist!"
      else
          # ffprobe returns height as 'height=xxx', filter out non numerics
          HEIGHT=$(ffprobe -v error -show_entries stream=height -of default=noprint_wrappers=1 $FILE | sed 's/[^0-9]*//g')
          echo 'Video height is '$HEIGHT
  
          # give warning resolution higher than video to be processed
          # pause for the first time then ignore all following warnings
          # to do
  
  
          # use shell parameter expansion, e.g.,
          # ~% FILE="example.tar.gz"
          # ~% echo "${FILE%%.*}"
          # example
          # ~% echo "${FILE%.*}"
          # example.tar
          # ~% echo "${FILE#*.}"
          # tar.gz
          # ~% echo "${FILE##*.}"
          # gz
  
          # settings according to google vp9 vod
          # https://developers.google.com/media/vp9/settings/vod/
  
          if [[ $RESOLUTION -ge 1080 ]]; then
  
              if [[ $HEIGHT -eq 1080 ]]; then
                  SCALE_FILTER_FULLHD="" # no scaling
                  echo 'No scaling needed'
              else
                  SCALE_FILTER_FULLHD="-vf scale=1920x1080:flags=lanczos"
                  echo 'filter setting: '$SCALE_FILTER_FULLHD
              fi
  
              # 1920x1080 60 fps 
              ffmpeg -y -i $FILE $SCALE_FILTER_FULLHD -b:v 3000k \
                     -minrate 2000k -maxrate 4350k -tile-columns 2 -g 240 -threads 8 \
                     -quality good -crf 31 -c:v libvpx-vp9 -c:a libopus -b:a 128k \
                     -pass 1 -speed 4 "${FILE%%.*}"-1080p.webm && \
                  ffmpeg -i $FILE $SCALE_FILTER_FULLHD -b:v 3000k \
                         -minrate 2000k -maxrate 4350k -tile-columns 4 -g 240 -threads 8 \
                         -quality good -crf 31 -c:v libvpx-vp9 -c:a libopus -b:a 128k \
                         -pass 2 -speed 1 -y "${FILE%%.*}"-1080p.webm
  
              if [[ $ALL == "n" ]]; then
                  continue
              fi
  
          fi
  
          if [[ $RESOLUTION -ge 720 ]]; then
  
              if [[ $HEIGHT -eq 720 ]]; then
                  SCALE_FILTER_HD=""
                  echo 'No scaling needed'                
              else
                  SCALE_FILTER_HD="-vf scale=1280x720:flags=lanczos"
                  echo 'filter setting: '$SCALE_FILTER_HD
              fi
  
              # 1280x720 30 fps
              ffmpeg -y -i $FILE $SCALE_FILTER_HD -b:v 1024k \
                     -minrate 512k -maxrate 1485k -tile-columns 2 -g 240 -threads 8 \
                     -quality good -crf 32 -c:v libvpx-vp9 -c:a libopus -b:a 128k \
                     -pass 1 -speed 4 "${FILE%%.*}"-720p.webm && \
                  ffmpeg -i $FILE $SCALE_FILTER_HD -b:v 1024k \
                         -minrate 512k -maxrate 1485k -tile-columns 2 -g 240 -threads 8 \
                         -quality good -crf 32 -c:v libvpx-vp9 -c:a libopus -b:a 128k \
                         -pass 2 -speed 2 -y "${FILE%%.*}"-720p.webm
  
              if [[ $ALL == 'n' ]]; then
                  continue
              fi
  
          fi 
  
          if [[ $RESOLUTION -ge 480 ]]; then 
  
              if [[ $HEIGHT -eq 480 ]]; then
                  SCALE_FILTER_SD=""
                  echo 'No scaling needed'                
              else
                  SCALE_FILTER_SD="-vf scale=640x480:flags=lanczos"
                  echo 'filter setting: '$SCALE_FILTER_SD
              fi
  
              # 640x480 30 fps
              ffmpeg -y -i $FILE $SCALE_FILTER_SD -b:v 750k \
                     -minrate 375k -maxrate 1088k -tile-columns 1 -g 240 -threads 4 \
                     -quality good -crf 33 -c:v libvpx-vp9 -c:a libopus -b:a 96k \
                     -pass 1 -speed 4 "${FILE%%.*}"-480p.webm && \
                  ffmpeg -i $FILE $SCALE_FILTER_SD -b:v 750k \
                         -minrate 375k -maxrate 1088k -tile-columns 1 -g 240 -threads 4 \
                         -quality good -crf 33 -c:v libvpx-vp9 -c:a libopus -b:a 96k \
                         -pass 2 -speed 2 -y "${FILE%%.*}"-480p.webm
  
              if [[ $ALL == 'n' ]]; then
                  continue
              fi
  
          fi
  
          if [[ $RESOLUTION -ge 360 ]]; then
  
              if [[ $HEIGHT -eq 360 ]]; then
                  SCALE_FILTER_MOBILE=""
                  echo 'No scaling needed'
              else
                  SCALE_FILTER_MOBILE="-vf scale=640x360:flags=lanczos"
                  echo 'filter setting: '$SCALE_FILTER_MOBILE                
              fi
  
              # 640x360 30 fps
              ffmpeg -y -i $FILE $SCALE_FILTER_MOBILE -b:v 276k \
                     -minrate 138k -maxrate 400k -tile-columns 1 -g 240 -threads 4 \
                     -quality good -crf 36 -c:v libvpx-vp9 -c:a libopus -b:a 96k \
                     -pass 1 -speed 4 "${FILE%%.*}"-360p.webm && \
                  ffmpeg -i $FILE $SCALE_FILTER_MOBILE -b:v 276k \
                         -minrate 138k -maxrate 400k -tile-columns 1 -g 240 -threads 4 \
                         -quality good -crf 36 -c:v libvpx-vp9 -c:a libopus -b:a 96k \
                         -pass 2 -speed 2 -y "${FILE%%.*}"-360p.webm
  
              if [[ $ALL == 'n' ]]; then
                  continue
              fi
  
          fi
  
          if [[ $RESOLUTION -lt 360 ]]; then
              echo 'Are you sure you want a low quality '$RESOLUTION'p video?!'
          fi
  
      fi
  
  done
  
  

• Mount and transfer files to Kindle. Mount the e-reader to a local path then copy files to documents folder.

  $sudo mount -t msdosfs /dev/msdosfs/Kindle ~/usbmnt/
  

• Eject Kindle.

  Unlike ejecting Kindle on a Linux machine using,

  $sudo eject /dev/disk/by-label/Kindle
  

  on FreeBSD, use

  $sudo camcontrol devlist
  <Kindle Internal Storage 0100> at scbus8 target 0 lun 0 (da4,pass5)
  

  or

  $sudo usbconfig
  ugen0.9: <Amazon Amazon Kindle> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (2mA)  
  

  to get the mounted device info first, then power off Kindle to keep reading while it is connected charging,

  $sudo usbconfig -d 0.9 power_off
  

  where “-d” is used for device selection with the format <unit>.<addr>, “0.9” stands for “[ugen]<unit0>.<addr9>”.

  To get vendor, product and other detailed information, use

  $sudo usbconfig -d 0.9 dump_device_desc
  

• Fcitx/Rime input method.

  The default installation does not include MSPY scheme. To add the input schema, go to .config/fcitx/rime, create a folder ./build and copy all files from /usr/local/share/brise here.

  There is a utility called rime_deployer. Compile the desired schema; copy generated *.bin file.

  $cd ~.config/fcitx/rime     # home directory for rime config
  $mkdir -pv build && cd "$_" # create 'build' dir and change dir to 'build'
  $cp -v /usr/local/share/brise/* ./                      # copy shared data
  $rime_deployer --compile double_pinyin_mspy.schema.yaml # compile schema
  $cp -v double_pinyin_mspy.prism.bin ../                 # copy bin file
  $cp -v double_pinyin_mspy.schema.yaml ../               # copy yaml file
  $rime_deployer --add-schema double_pinyin_mspy # generate default.custom.yaml
  

• Inkscape crash immediately after launch.

  See bugzilla report by many. The fix now is compile and install /usr/ports/devel/glib20 and #pkg lock glib.

• How to insert \(\LaTeX\) math into a video.

• Mount NTFS device on FreeBSD.

  We need fusefs-ntfs driver,

  # pkg install fusefs-ntfs
  

  Also make sure in /boot/loader.conf, there is a line fuse_load="YES".

  Use gpart show to see the device name of storage disk. In the example below, we mount the NTFS slice with standard Linux permissions,

  # ntfs-3g -o permissions /dev/da4s2 ~/usbmnt/
  

• Copy files with progress and speed information.

  The trick is to use FILE protocol of curl for local files.

  $curl -o ~/Downloads/file FILE:///path/to/file/on/storage/file
  

• Switch to bash.

  From handbook/shells, switch to bash. This should not be done with root. Leave the root’s shell as tsch.

  % chsh -s /usr/local/bin/bash
  

  .bash_profile loads .profile and loads .bashrc if the shell is interactive. Put environment variables, session settings in .profile and aliases, completion etc., in .bashrc.

  # .bash_profile
  . ~/.profile
  if [[ $- == *i* ]]; then . ~/.bashrc; fi
  

  Comment out (set-exec-path-from-shell-PATH)) in general settings for emacs since it hangs GUI.

  ; set PATH, because we don't load .bashrc
  ; function from https://gist.github.com/jakemcc/3887459
  (defun set-exec-path-from-shell-PATH ()
    (setenv "PATH" (concat "/usr/local/bin:" (getenv "PATH")))
    (let ((path-from-shell (shell-command-to-string "$SHELL -i -c 'echo -n $PATH'")))
      (setenv "PATH" path-from-shell)
      (setq exec-path (split-string path-from-shell path-separator))))
  
  ;(if (window-system)
  ;        (set-exec-path-from-shell-PATH)) ; GUI hangs in FreeBSD
  

• Grow zfs pool.

  The original zpool were four HITACHI C10K900 300G disks. They are replaced with four HITACHI C10K1800 600G disks.

  Disk S/N	zpool label
  KLV0Z8GF	zfs0
  KLV0ZYZF	zfs1
  KLV0YAVF	zfs2
  KLV0YN9F	zfs3

  The procedure is to replace disks one by one.

  • Remove an old disk from zpool before power on.
  • Power on; check the missing disk info by zpool status. Write down the labels according to gpart backup da0 for example.

  • Copy partition geometry from mirror disk to new disk

    # gpart backup da1 | gpart restore -Fl da3
    

  • Modify the labels

    # gpart modify -i 1 -l efiboot0 da3
    # gpart modify -i 2 -l gptboot0 da3
    # gpart modify -i 3 -l swap0 da3
    # gpart modify -i 4 -l zfs0 da3
    

  • Clone EFI and GPT boot partitions

    # dd if=/dev/da1p1 of=/dev/da3p1
    # dd if=/dev/da1p2 of=/dev/da3p2
    

  • Resilver the new disk

    # zpool replace zroot uid-of-missing-disk da3p4
    

  • Upgrade zpool if action says so

    # zpool upgrade zroot
    

• Compile GoldenDict on FreeBSD.
  • Check out git repository
  • Install qt5-qmake, qt5-help and qt5-linguisttools.

  • Enter folder goldendict and qmake without epwing or audio support

    qmake "CONFIG+=no_ffmpeg_player" "CONFIG+=no_epwing_support"
    

  • Modify bgl_babylon.cc and processwrapper.cc according to

    https://lists.freebsd.org/pipermail/svn-ports-head/2013-September/028961.html

    https://svnweb.freebsd.org/ports?view=revision&revision=327062

    The changes in the repository are as follows,

    diff --git a/bgl_babylon.cc b/bgl_babylon.cc
    index 80c3b6b..87643ee 100644
    --- a/bgl_babylon.cc
    +++ b/bgl_babylon.cc
    @@ -27,6 +27,7 @@
     #include <stdlib.h>
     #include <string.h>
     #include <stdio.h>
    +#include<unistd.h> // added for freebsd
     #include <iconv.h>
     #include <QTextDocument>
     #include "gddebug.hh"
    diff --git a/processwrapper.cc b/processwrapper.cc
    index 9d52583..7d1f31e 100644
    --- a/processwrapper.cc
    +++ b/processwrapper.cc
    @@ -1,5 +1,6 @@
     #include "processwrapper.hh"
    
    +#include <unistd.h> // added for freebsd
     #include <QtCore>
    
     #ifdef Q_OS_WIN32
    

  • Add to Makefile line LIBS, -lexecinfo -liconv.

  • make -j8 and test run ./goldendict.

    

  • iconv issues related to UTF-8

    https://lists.freebsd.org/pipermail/freebsd-questions/2015-October/268672.html

    https://lists.freebsd.org/pipermail/freebsd-ports/2015-July/099680.html

    The major problem here is GoldenDict won’t index Babylon *.bgl correctly. Sometimes it complains UTF-8 head is not correct; sometimes it complains nothing but won’t index the dictionary anyway. Might well just try the FreeBSD native iconv with adding -DLIBICONV_PLUG to CXXFLAGS in Makefile.

• Open *.djvu with Evince. Multipage *.djvu file is not correctly identified. See bug report for more.

  File type DjVu image (image/vnd.djvu) is not supported
  

  The fix is given by Comment 3 down below on bug report page. On FreeBSD, the path is

  /usr/local/lib/evince/4/backends/djvudocument.evince-backend
  

  Add image/vnd.djvu to the last line MimeType= in djvudocument.evince-backend.

• Upgrade to 11.2-Release.
  • Back up configuration files.

  • Follow the release installation page and use

    # freebsd-update upgrade -r 11.2-RELEASE
    

    freebsd-update install will be followed, after reboot and be applied again.

  • If packages are managed by pkg, reinstall all packages

    # pkg-static upgrade -f
    

    See more in handbook/updating-upgrading.

  • To fix broken nvidia drivers (ABI linked to older kernel version), we need to install /usr/src and use /usr/ports to build driver ourselves.
    • Get src.txz from http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.2-RELEASE/

    • Unpack (and keep the source tree)

      # tar -C / -xvf src.txz
      

      -C will change working directory to root, then tar unpacks with verbose mode for file src.txz.

    • Refresh /usr/ports

      # portsnap fetch update 
      

    • Go to /usr/ports/x11/nvidia-driver-340 and build from source

      # make install clean
      

      clean coming after install will clean working directory. We might want to follow OpenSSL Wiki to add to /etc/make.conf

      DEFAULT_VERSIONS+= ssl=openssl
      

    • If pkg complains about changed configuration of nvidia driver, lock the pkg version with

      # pkg lock nvidia-driver-340
      

• After upgrading to emacs 26.1, the Gnus+Gmail log in stopped working since it complained gpg could not be found. Basically pinentry-tty cannot work properly with newer Emacs. So we need to “loop back” to older pin entering scheme, add the following to init.el

  (setf epa-pinentry-mode 'loopback) % freebsd only 
  

• Change pkg repository to the latest from quarterly, modify the file /etc/pkg/FreeBSD.conf or follow the instruction inside to create a local copy at /usr/local/etc/pkg/repos/FreeBSD.conf.

• Using tor with regular user switching to _tor. After enabling control port 9051, if not using root or _tor, nyx will complain control_cookie_auth does not exist, since it is in /var/db/tor/ which is owned by _tor.

  A temporary fix to this problem is to use visudo to add a regular user with the power of issuing nyx as user _tor without a password.

  someuser ALL=(_tor) NOPASSWD: /usr/local/bin/nyx
  

• Regarding *.core dump, see discussion on forums.freebsd.org. It seems we can disable core dump by setting

  kern.coredump=0
  

  or

  kern.corefile=/dev/null
  

  in /etc/sysctl.conf.

• Use Firefox with profiles. There are two methods to create new profiles.

  • With no firefox instances running (including running background), start firefox with firefox -p.
  • With firefox running, type about:profiles into address bar and follow the About Profiles page.

  To start firefox with a specified profile, use firefox -p NAMEofProfile. See here for more.

• Install security/tor from ports and enable tor web browsing through regular Firefox

  Preferences –> Network Proxy –> settings.

  Set socks5 proxy 127.0.0.1 port 9050. Also check socks5 proxy for DNS.

  But we need to skip loop back network interface lo0 for localhost. (Use ifconfig and check if the interface has IP 127.0.0.1 for localhost.)

  Add set skip on lo0 in /etc/pf.conf after block all.

• Mount remote server directory. Install first sshfs of course! To allow a normal use mount and check out the mounted remote directory, add

  vfs.usermount=1
  

  to /etc/sysctl.conf.

  A normal user cannot open a FUSE device. Change this by

  # devfs ruleset 10
  # devfs rule add path 'fuse*' mode 666
  

  When mounting a remote directory use

  # sshfs -o allow_other USER@host:/path/to/mount/ /local/mount/point/
  

• Show UTF-8 characters in terminal. Set locale in .cshrc

  setenv LANG en_US.UTF-8
  setenv MM_CHARSET UTF-8
  

• Auto complete with sudo. Add to .cshrc

  set complete = enhance
  complete sudo 'n/-l/u/' 'p/1/c/' 
  

  More on this IBM link.

• TeXLive DVD does not contain binaries for architecture amd64-freebsd but the online installer install-tl does.

  To use graphic installer, install p5-Tk from ports.

  xelatex needs libfreetype.so.9. Force a symlink

  # ln -s /usr/local/lib/libfreetype.so.6.14.0 /usr/local/lib/libfreetype.so.9
  

• Mount ISO images

  First attach the image to unit #1

  # mdconfig -o readonly -a -t vnode -f /path/to/.iso or .img -u 1
  

  Then mount with type -t, read-only

  # mount -o ro -t cd9660 /dev/md1 /mnt
  

  Unmount unit #1 using

  # umount /mnt
  # mdconfig -d -u 1
  

• One quirky thing about FreeBSD emacs-nox is undo is not bounded to Ctrl-/, but the beginner friendly Ctrl-x u.

  Another weird thing is backspace will invoke help Ctrl-h ?. Use the following to recover the erase key.

  (normal-erase-is-backspace-mode 1) ; for freebsd emacs
  

• LXDE on FreeBSD by default uses openbox as window manager. The openbox configuration is purely text based and its location is /.config/openbox/lxde-rc.xml.

  To start LXDE automatically with startx, add

  % echo "exec startlxde" > ~/.xinitrc
  

• If Dbus complains about “no machine-id”, use

  # dbus-uuidgen > /etc/machine-id
  

• During installation, there is an option whether invite new user to other group. Say yes and add the newly created user to wheel group. Users in wheel group can escalate user privilege to root.
• After installed sudo, only need to uncomment out the line for wheel group; no need to add new user to sudo group.

• There are three important files regarding WiFi connection with static IP. /etc/rc.conf, /etc/resolv.conf and /etc/wpa_supplicant.conf. For connection to a hidden SSID, use configuration in /etc/wpa_supplicant

  ...
  network={
  ssid="/ssid name/"
  scan_ssid=1
  priority=5
  bssid="/mac addr of AP/"
  pairwise=CCMP
  key_mgmt=WPA-PSK
  proto=RSN
  psk="/password/"
  }
  ...
  

  In /etc/rc.conf, add channel number helps fast establishment of static IP connections

  ...
  ifconfig_wlan0="WPA inet 192.168.xxx.xxx netmask 255.255.255.0 ssid /ssid name/ ch
  annel 48:ht/40"
  ...
  

  Sometimes need to manually connect.

  %sudo service netif stop  # stop network 
  %sudo service netif start # restart
  

• To kill zombie logged in users, use w or who to find out their login types. Then use ps auxw | grep -i tty0 to find out PID. Use kill PID to kick out those users.

• pkg primer, https://wiki.freebsd.org/PkgPrimer. /usr/ports itself provides searching mechanisms. cd /usr/ports then make index or make fetchindex. make search key=WORD will provide the searching functionality.

  The most commonly used commands,

  $sudo pkg update               # refresh software repo
  $sudo pkg upgrade              # update all outdated packages
  $sudo pkg lock PACKAGE         # prevent PACKAGE from upgrading
  $sudo pkg clean -a             # clear cached packages
  $sudo pkg info                 # list all installed packages
  $sudo pkg info PACKAGE         # list detailed info about PACKAGE
  $sudo pkg info -D PACKAGE      # show post installation message 
  $sudo pkg delete PACKAGE       # uninstall PACKAGE (and those depending on it)
  $sudo pkg delete -f PACKAGE    # uninstall PACKAGE alone 
  $sudo pkg autoremove           # uninstall packages no longer needed
  $sudo pkg search PACKAGE       # search for PACKAGE
  

• View message of an installed package. Sometimes it contains important post installation howto. Installed packages are cached at /var/cache/pkg/.

  %pkg info --pkg-message PACKAGE
  

  To update FreeBSD itself, use freebsd-update fetch followed by freebsd-update install.

• Networking checking. netstat -r will show route table. And to check listening port, we can use

  %netstat -an | grep -i listen
  

  sshd processes can be part of output of sockstat -4l. Connection authorization log is accessible via /var/log/auth.log.

• Install devcpu-data. Then check if there is a new microcode update for CPU.

  %cpucontrol -u -v /dev/cpuctl0
  

A newcomer and potential competitor with C++.

• Install KOReader on Kindle Touch 3G (K5).

  • Jailbreak Kindle with Kubrick
  • Install KUAL and reinstall accompanied MKK
  • Install KOReader

  Launch KOReader through KUAL, from top menu,

  • Second Tab : Switch zoom mode -> zooming to fit content width
  • Third Tab : Status bar -> only keep current page, pages left in chapter and progress percentage
  • Third Tab : Screen -> E Ink settings -> check avoid black flashes in UI

  From bottom menu, set reading in landscape mode.

  Copy stardict dictionaries to koreader/data/dict directory. It seems dictionaries containing CJK characters would crash KOReader and make kindle reboot. Also it takes fairly long time for search results to pop up if large dictionaries are used for lookup.

  As for Wikipedia lookup, if multiple languages are listed, at the moment the lookup results will be shown in languages looping over the list of specified languages. For example, if we tell KOReader to query Wikipedia pages in en de fr jp zh five different languages, first we will be shown search results in English, press the button at bottom center en > de, the search results in German come next; the button itself changes to de > fr. Press the button again, the search results in French turn up and etc. In this example, wouldn’t it be nice if pressing the button en > de, a drop down list consisting of five different languages pops up instead of looping over the entire language list?

• Clonezilla larger disk to smaller disk.

  This scenario is not supported by Clonezilla. The workaround is shrink all the larger disk partitions so that the total space in use is less than or equal to the size of the smaller disk. This can be done with gparted live CD. We cannot gparted resize mounted partitions on a running Linux machine. Shut it down and resize in live CD environment instead.

  Then boot into Clonezilla GUI with its live CD,

  • Do a larger disk partitions to image backup, use SSH server if no spare disks are available;
  • Do a partitions image to smaller disk restore. Keep the original partition table and do not check resize the geometry of partition table to fit target.

  Once everything is done, boot into gparted live again to max out the capacity of the smaller disk by resizing whatever is allowed.

  When booting into the smaller disk with newly restored system, the device UUIDs might be different, use blkid to print the new UUIDs and update /etc/fstab accordingly. Lastly,

  $sudo update-initramfs -u 
  

  to reflect the updated UUIDs. We might need to update the swap UUID in /etc/initramfs-tools/conf.d/resume as well.

• A trick to convert capital letters in file name to lowercase.

  In Bash 4+, use shell parameter expansion.

  $a='string WITH CAPS'
  $echo $a
  string WITH CAPS
  $echo ${a,,}
  string with caps
  $echo ${a,,[C-W]}
  string with cAps
  

  Besides, , for uppercase to lowercase; ^ for lowercase to uppercase.

• Add leading zeros to filenames.

  for f in [0-9].pdf; do
  > mv -v $f $(printf %02d.%s ${f%%.*} ${f##*.})
  > done
  
  renamed '0.pdf' -> '00.pdf'
  renamed '1.pdf' -> '01.pdf'
  renamed '2.pdf' -> '02.pdf'
  renamed '3.pdf' -> '03.pdf'
  renamed '4.pdf' -> '04.pdf'
  renamed '5.pdf' -> '05.pdf'
  renamed '6.pdf' -> '06.pdf'
  renamed '7.pdf' -> '07.pdf'
  renamed '8.pdf' -> '08.pdf'
  renamed '9.pdf' -> '09.pdf'
  

  To combine them, use

  $pdftk *.pdf cat output OUTPUT.pdf
  

• Gzip nginx module.

  Basic web server only compress html; so if we want to server compressed text, images and other media types, we need to configure the nginx server to tell the differences between different files.

  See

  https://gist.github.com/kilhage/7f0e7546457716dc9174

  https://guides.wp-bullet.com/enable-svg-gzip-compression-nginx/

  and an article on digitalocean.

  server {
      # inside /etc/nginx/sites-available/active-website
      ... 
  
      gzip on;                   # turn on 
      gzip_comp_level    5;      # 1 - 9; 9 most, 1 least 
      gzip_min_length    256;    # size less 256 won't be compressed
      gzip_proxied       any;    # for request routed via proxy servers 
      gzip_vary          on;     # for non gzip browsers
      gzip_buffers       32 16k; # 32 buffers with each of the size 16k 
      
      gzip_types
      application/atom+xml
      application/javascript
      application/json
      application/ld+json
      application/manifest+json
      application/rss+xml
      application/vnd.geo+json
      application/vnd.ms-fontobject
      application/x-font-ttf
      application/x-web-app-manifest+json
      application/xhtml+xml
      application/xml
      font/opentype
      # woff is already compressed 
      image/bmp
      image/svg+xml
      image/x-icon
      # jpeg is already compressed
      text/cache-manifest
      text/css
      text/plain
      text/vcard
      text/vnd.rim.location.xloc
      text/vtt
      text/x-component
      text/x-cross-domain-policy;
      # html is always compressed by gzip module
  }
  

  Add the above settings then do a test sudo nginx -t before reloading nginx server. To test if gzip functions correctly, use web browser debug tool. For example, with firefox Web Developer -> Network (Ctrl-Shft-E), look for a media type instructed to be compressed by gzip and check Response headers. There should be a field

  Content-Encoding: gzip
  

• Web font.

  If not importing web fonts from CDN (Content Delivery Network) services such as Google Fonts, put desired fonts to local server and use @font-face in CSS sheet to load them. As to how to optimize the web fonts on a website, see here for detailed discussions.

  There are generally four major font types: *.woff2, *.woff, *.ttf and *.eot. There are listed here in a descending order in terms of how long they have been around. WOFF is the most commonly used while the two older ones are intended for legacy support for Android (4.4 or below) and Internet Explorer (IE 9 or below) respectively.

  The newest WOFF2 is a compressed version of WOFF. It seems FontForge at the moment has difficulty supporting conversion of available fonts to WOFF2. The temporary fix is install woff2 package. It comes with both compress and decompress utilities (for conversion between TTF and WOFF2).

• Create patches and apply patches.

  To create a patch for a single modified file,

  $diff -Nau original-file modified-file > patch-file
  

  where -N means treat absent file as empty; -a means treat files as text; -u means unified format.

  To apply a patch,

  $patch original-file < patch-file 
  

• Some text processing.

  To extract text between double quotes, we can use cut, where -d indicates the delimiter while -f2 indicates the second field.

  $cut -d '"' -f2 < /path/to/original > /path/to/processed
  

  To delete a line containing specific pattern, we can use sed

  $sed '/PATTERN/d' originalfile
  

  This will print process file to standard out. With -i option, it will modify the file (in-place edits)

  $sed -i '/PATTERN/d' originalfile
  

  With i.bak, it will create a backup file.

• Set Gnome workspace names.

  $gsettings set org.gnome.desktop.wm.preferences workspace-names "['0', '1']"
  $gsettings get org.gnome.desktop.wm.preferences workspace-names
  ['0', '1']
  

• Install VMware tools without X.

  • After clicking “Install tools”, the *iso has been loaded to device /dev/cdrom. Mount it

    $sudo mount /dev/cdrom /mnt
    

    then unpack tarball. Execute vmware-install.pl.

  • The recommended way though is install open-vm-tools from repository.

• Use ionice for maintenance of btrfs.

  For scrub,

  $sudo ionice -c idle btrfs scrub start /
  

  For defragment,

  $sudo ionice -c idle btrfs filesystem defragment -f -t 32M -r SOMEPATH
  

  • -f: Flush data before move on to next file.
  • -r: Recursive defrag current path.
  • -t 32M: Chunk size 32M is recommended size, see this thread for more information.
• In Emacs Dired mode, batch rename either
  • by edit mode, toggle C-x C-q then rectangular edit; or
  • by % R after marking files, regular expressions are used. See manual for more.

• lxterminal color fix. The solarized theme renders an unusable coloring of emacs -nw. To have a better color rendering, add to .bashrc (not a good idea though)

  $export TERM=xterm-256color
  

• Wipe SSD/HDD.

  First round, use the generic shred with 3 loops of random fill, turn on verbose message,

  $sudo shred -v /dev/sdX
  

  Second around, use ATA Secure Erase (reset SSD cells). Here note the trick—most desktop PC once operating system is fully up, hard drive will be “frozen”. To unfroze, suspend to RAM (BIOS S3 settings) then back to power again. For example, this can be done on a ThinkPad with Debian live CD.

  Last round, use shred again but a round of zero fill, no repeats.

  $sudo shred -v -n 1 -z /dev/sdb
  

• ath9k 5GHz. Set REGDOMAIN=US in /etc/default/crda. Verify

  $sudo iw reg get
  

  To check if 5GHz bands are available,

  $sudo iw list | grep -i mhz
  

• Working with git from emacs

  http://alexott.net/en/writings/emacs-vcs/EmacsGit.html

• Compile emacs from GNU source. The reason of building from source is Debian team is still working on porting the latest emacs one month after stable emacs was released.

  https://www.emacswiki.org/emacs/EmacsFromGit

  https://www.emacswiki.org/emacs/BuildingEmacs

  https://www.emacswiki.org/emacs/EmacsSnapshotAndDebian

  • Only checkout the latest stable branch emacs-26.

    $git clone --depth 1 --branch emacs-26 git://git.sv.gnu.org/emacs.git
    

  • Enter emacs folder and execute

    $./autogen.sh  # generating compiling files
    $./configure --prefix=/home/USER/engrsoft/emacs/ --with-mailutils 
    

  • Install whatever ./configure complains of lacking of. But do install

    $sudo apt-get install libgnutls28-dev libm17n-dev # for encrypto and nonlatin char
    

    Regarding libgtk2.0-dev, without it, it seems it still builds but emacs ends up with a lucid GUI rather than GTK+.

    $MAKE='make -j4' make bootstrap # build with multi job workers 
    $make install                   # non root install, check out --prefix= folder
    

  • To keep up with the latest branch source,

    $git clean -dxf  # cleaning up old files
    $git pull 
    $./autogen.sh
    ...
    

• Pulse Audio over ssh.

  There are generally two methods. The first one is install (server side) paprefs and enable Network Server –> Enable network access to local sound devices.

  The second method is again on server side

  export PULSE_SERVER="tcp:localhost:14713"
  

  where 14713 is the port from client side used to access pulse audio server. For example, from client side, connect with X forwarding

  ssh -C -R 14713:localhost:4713 -X USER@SERVER 
  

  where -R gives TCP port binding and -C offers data compression. 4713 is the default pulse audio server port. Port 14713 in this example needs to be added to port forward rule in firewall (OPNsense).

• top / htop like traffic monitoring tools. Installed iptraf. Need sudo to invoke.

• In case of VLC not being able to play files on a server, use sshfs to mount the remote directory locally.

  $sshfs USER@host:/path/to/mount/ /local/mount/point/
  

  To unmount mounted directory, use fusermount -u /local/mount/point/.

• “Refresh” file equivalent of emacs, Ctrl-x Ctrl-v RET it is really “find alternative file” (letter v vs. letter f).

• Check patch status against Intel Meltdown and Spectre.

  $tail /sys/devices/system/cpu/vulnerabilities/*
  ==> /sys/devices/system/cpu/vulnerabilities/meltdown <==
  Vulnerable
  
  ==> /sys/devices/system/cpu/vulnerabilities/spectre_v1 <==
  Mitigation: __user pointer sanitization
  
  ==> /sys/devices/system/cpu/vulnerabilities/spectre_v2 <==
  Mitigation: Full generic retpoline - vulnerable module loaded
  

• vi quick reference card. *.pdf version. Cheat Sheet. Another Cheat Sheet. Solaris advanced user guide on vi and search and replace in vi.

  Why learning vi given I am an Emacs user? vi is most suitable for system administrative tasks; Emacs is for more general writing. The major incentive is I am not happy with nano when managing linux boxes.

  • Type v to turn on visual mode then everything that is hightlighted can be yanked.
  • gg sends the cursor to the beginning of the file and G sends the cursor to the end. However if jumping multiple lines is desired, use NUM M combination, where NUM is a number and M is a movement.
  • set: number turns on line numbers.

  • To enter the same prefix to multiple lines.

    • Move the cursor to the beginning of the first of those lines;
    • Ctrl-v enters the visual mode;
    • j multiple times depending on the number of lines;
    • I enters insert mode (note this is upper I) and enter prefix;
    • Esc to exit.

    An alternative is to enter prefix for one line and use ., i.e., repeating the last edit.

    The Emacs equivalent is rectangle selection which is more convient:

    • Activate mark, Ctrl-n \(m\) rows and Ctrl-f \(n\) columns to create an \(m \times n\) rectangle.
    • It is followed by Ctrl-x r t for changing text contents.

• Convert .pdf graphics to *.jpg or *.png with Linux without using Adobe Acrobat Professional.

  $convert -density 600 -trim PICS.pdf -quality 100 PICS.jpg
  

  Conversion flags:

  • -density controls dpi
  • -trim removes unwanted edge pixels
  • -quality specifies the compression level

• Change the default booting kernel.

  $sudo vi /etc/default/grub # change GRUB_DEFAULT=0 to "1>2" e.g. 
  $sudo update-grub
  

  "1>2" selects grub main menu item 1 (0-based, 0 is the first item) and submenu item 2. Usually, the latest kernel is submenu item 0 and submenu item 1 is its recovery mode. Main menu item 0 is the default booting kernel and main menu item 1 is the advanced menu entry.

• Reformat OEM SAS drives with block size 512b. IBM rebranded Hitachi SAS drives use custom firmware and block size 528. To revert it back to retail version with block size 512, use sg3-utils.

  $sudo install sg3-utils                             # install SCSI utilities 
  $sudo sg_scan -i                                    # scan devices
  $sudo sg_format --format --size=512 --six /dev/sgx  # --six for mode select
  

• Disable Gnome 3 auto start at boot. On Debian 9, this can be achieved by disabling gdm.

  $sudo systemctl set-default multi-user.target # login via console, gdm not started
  $sudo systemctl get-default                   # default was graphical.target
  

• Cross flash IBM ServeRAID M5015. Search for old firmware and MegaRAID Storage Manager. Use legacy product group and product name 9260-8i.

  FreeDOS>megarec.exe -cleanflash 0          # erase, 0 is the card index
  FreeDOS><SHUTDOWN and REBOOT>
  FreeDOS>megarec.exe -m0flash 0 v167.rom    # flash version 167 ROM
  FreeDOS>megarec.exe -writesbr 0 sbrlsi.bin # switch banner to LSI from IBM
  

• Update firmware of IBM ServeRAID H1110.

  FreeDOS>sas2flsh.exe -o -e 6            # erase, -o advanced mode
  FreeDOS>sas2flsh.exe -o -f 2114it.bin   # switch to IT mode/IR mode, use 2114ir.bin
  FreeDOS>sas2flsh.exe -o -b mptsas2.rom  # flash RAID bios
  FreeDOS>sas2flsh.exe -list              # check info
  

• One problem with btrfs RAID array is, if there are devid holes and the physical disks are of the same size, it is hard to tell which devid is associated with which disk in the physical disk bay. Here comes the utility lsscsi. Install it from package repo with the same name and use

  $lsscsi -v
  

  to read detailed information about each disk such as brand name, model, mount point, and firmware etc.

  On ASRock J3455-ITX, lsscsi -v [0-3:x:x:x] correspond to SATA 1-2, SATA A1-A2.

  lsscsi output	physical port
  [0:x:x:x]	SATA 1
  [1:x:x:x]	SATA 2
  [2:x:x:x]	SATA A1
  [3:x:x:x]	SATA A2

• Resilver btrfs RAID array. (No more dd.)

  $sudo btrfs replace start /src/dev/ /target/dev/ /btrfs/array/mnt/
  

  If replace start subcommand complains about existing file system on the new device, we need to use -f force flag after start. The progress of resilvering can be accessed by

  $sudo btrfs replace status /btrfs/array/mnt/ # use Ctrl-C to exit viewing progress
  

  To max out the capacity of new device, use resize. See SUSE doc for more examples.

  $sudo parted                # enter GNU parted prompt
  (parted)print               # list partitions on current device 
  (parted)resizepart NUMBER   # enter the partition number 
  (parted)END                 # enter end sector then exit 
  $sudo btrfs filesystem resize <dev_id>:max /btrfs/array/mnt/
  

  For root-on-btrfs, i.e., replacing the disk which hosts the /boot/ partition, consider advanced mode -k1 with Clonezilla. (Shut down server and clone the first disk using Clonezilla Live CD with new disk USBed to server, for example.)

  Finally, scrub the new array with balanced I/O load,

  $sudo ionice -c idle btrfs scrub start /btrfs-mount-path/
  

• Find corrupt files associated with csum. The normal procedure to recover and rescue a broken btrfs system is outlined on its wiki page for btrfsck.

• Remove old core packages from snap.

  $sudo snap remove core --revision 1234
  

• Check Physical Drive information about SAS disks behind hardware LSI RAID controller.

  $sudo megacli -AdpGetPciInfo -aAll # get adapter info
  $sudo megacli -LdPdInfo -a0        # get drive info
  

• Add/delete new user.

  $sudo adduser USER               # add a new user; follow questionnaire
  $sudo usermod -aG sudo USER      # add USER to 'sudo' group with privileges
  $sudo deluser --remove-home USER # delete a new user and assoc. home dir
  

• Nextcloud command line mode.

  $nextcloudcmd -u USER -s LOCAL_FOLDER https://127.0.0.1/remote.php/webdav/
  

  Note that there is no need to specify /USER folder after webdav. s flag stands for silence, i.e., no verbose diagnostics.

  See more details here.

• Install suckless surf and tabbed. They are available from Debian official repository. Do install the version from testing repo though.

  $sudo apt-get install suckless-tools
  $sudo apt-get install surf
  

  Add alias websurf='tabbed -c surf -pe' to .bashrc and start web surfing with websurf. The -c option for tabbed is for enabling the same tab behavior of modern browsers, i.e., close tab to quit; p option for surf for disabling plugins. To read more from man pages

  $man surf
  $man tabbed
  

  • Ctrl-Shft-RET: Create new tab.
  • Ctrl-Shft-h or l: Switch between tabs.
  • Ctrl-j or k: Scroll down and up.
  • Ctrl-q: Quit a tab.
  • Ctrl-g: Enter URL.
  • Ctrl-h: Go back in history.
  • Ctrl-y: Yank the link of current page or the page over whose link the cursor hovers.
  • Ctrl-p: Load the link yanked with Ctrl-y.

  Easter egg: Try Alt-x on Wikipedia; it will load a random page.

  To use surf as the default browser to preview local *.html’s, delete mimeapps.list and mime folder,

  $cd ~.config
  $rm -iv mimeapps.list # or use 'mv -v' instead
  $cd ~.local/share/applications
  $rm -iv mimeapps.list
  $cd ../ # now in ~.local/share again
  $rm -rf mime
  

• Fix partition mislabelling (TODO: this works for partition fix, but partition is not the same as device. Does it work in my case?) Say the original order of labelling of the installed btrfs system has the following structure,

  devid  1  ... /dev/sda
  devid  2  ... /dev/sdb
  devid  3  ... /dev/sdc
  devid  4  ... /dev/sdd
  

  We want to swap /dev/sdc for a new device, so we do the following:

  • Delete /dev/sdc from current RAID 1 array and balance among the rest three disks.
  • Install a new /dev/sdc and balance among all four disks.

  The problem is the newly installed /dev/sdc will be assigned a new devid \(5\) rather than the original \(3\). This can be fixed by fdisk using x (extra features) followed by f (fix partition order). Confirm operation with w (write changes).

• git log takes flags -p (display detailed commits) and -n (the most recent \(n\) commits. Also commits can be viewed by specifying “by date” flags,

  git log --after='2017-11-15' --before='yesterday'
  

  git log can be replaced with git shortlog if commits summary is desired.

  To reset, use git checkout . (. is not the period of this sentence.)

• Firefox font rendering patch. Create a file .fonts.conf under ~/ and paste the following as its contents.

  <?xml version='1.0'?>
  <!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
  <fontconfig>
   <match target="font">
    <edit mode="assign" name="rgba">
     <const>rgb</const>
    </edit>
   </match>
   <match target="font">
    <edit mode="assign" name="hinting">
     <bool>true</bool>
    </edit>
   </match>
   <match target="font">
    <edit mode="assign" name="hintstyle">
     <const>hintslight</const>
    </edit>
   </match>
   <match target="font">
    <edit mode="assign" name="antialias">
     <bool>true</bool>
    </edit>
   </match>
    <match target="font">
      <edit mode="assign" name="lcdfilter">
        <const>lcddefault</const>
      </edit>
    </match>
  </fontconfig>
  

• Change default web browser to firefox. To check current priorities of installed web browsers,

  $sudo update-alternatives --config x-www-browser
  There are 2 choices for the alternative x-www-browser (providing /usr/bin/x-www-browser).
  
    Selection    Path                                    Priority   Status
    -------------------------------------------------------------
  * 0            /usr/local/bin/firefox                   220       auto mode
    1            /usr/local/bin/firefox                   220       manual mode
    2            /usr/bin/surf                            30        manual mode
  

  To add more browsers to the list,

  $sudo update-alternatives --install /usr/bin/x-www-browser x-www-browser /path/to/new/browser 250
  

  The last argument 250 is the priority. /usr/bin/sensible-browser will check $BROWSER, gnome-www-browser (if Gnome in use), x-www-browser, and www-browser in order. See discussion for more.

• Change firefox update channel from Beta to Release. Go to /FIREFOX_ROOT/defaults/prefs/, edit channel in channel-prefs.js file.

  On Mac, it is in /Applications/Firefox.app/Contents/Resources/defaults/prefs/.

• Specify port number in remote connection
  • For ssh, use ssh -p 1234 user@site. (Note: type ~. to disconnect a session.)
  • For scp, use scp -P 1234 /files/or/folders user@site:/path/to/folder/. Note the upper case -P.
  • For sftp, use sftp -p -P 1234 user@site. (Note: -p is preserving time stamp.)

  • For sftp connection in vlc network streaming, use

    sftp://192.168.2.1:22/home/user/Videos/video.mp4

• Change display manager from default wayland to gdm, comment out WaylandEnable=false in /etc/gdm3/daemon.conf.

• Enter UTF-8 characters using US International Keyboard. For example, enter ğ by the following key sequence: Alt Gr-Shft u-g. Alt Gr-Shft u-Shft g will insert the upper case Ğ.

  Also check out Unibyte-Mode of emacs. Use C-x 8 C-h to list all defined built-in characters in emacs.

• ThinkPad X41 with X41T motherboard installed boots then gets stuck right before graphic display shows up. It seems to be a kernel bug when initializing wacom touch screen. Delete xserver-xorg-input-wacom then X41 boots into LXDE without problems.

• ThinkPad fan control thinkfan setting,

  $sudo thinkfan -q -b-3 -s 3 # quiet mode
                              # bias -3 (even out temp spike)
                              # update interval 3 seconds
  

• Check battery information from command line

  $cat /sys/class/power_supply/BAT0/status
  

  All available options are stored in /sys/class/power_supply/BAT0, e.g., serial_number, charge_full, charge_full_design etc.

• Laptop mode and tp_smapi for advanced power management. Check ArchWiki here and here. ThinkPad X41 specific page.

  $sudo vi /etc/modules
  # add thinkpad_ec tp_smapi hdaps
  

• Use nvlc on headerless server together with tmux.

  $tmux new vlc # new named session 'vlc'
  $nvlc         # start vlc player 
  

  Once in VLC player command line interface, type h for help. Common operations include

  • B: Open file browser.
  • Space: Play/Add files.
  • Shft-P: Show play list.

  Press Ctrl-b then d to detach a tmux session. We will be returned to terminal. If later we want to reattach to the existing session, use tmux a -t 0 where a means attach, -t specifies target and 0 is a default unnamed tmux session. Use tmux ls to see all active sessions.

• To enable minor modes in Emacs when opening a file , use -hook’s rather than specifying local variables at the end of the file. See details in the manual.

• Recently, a change in coreutils-8.2x has made ls wrap within quotes any file whose name contains spaces.

  $ls
   'file name with spaces.txt'
  $ls
   file_name_with_outout_spaces.txt
  

• Use tar to create *.xz archives, which is opposite to tar xavf file.tar.xz.

  $tar -cvJf /path/to/target.tar.xz /path/to/source/files
  $man tar
   -c, --create
         create a new archive
   -v, --verbose
         verbosely list files processed
   -J, --xz
   -f, --file ARCHIVE
         use archive file or device ARCHIVE
  

  Create an *.xz tar ball with pipe mode. Obviously this mode allows multithreading of xz.

  $tar c /some/dir | xz -4T0 > name.tar.xz     # Linux
  $tar -cf - /some/dir | xz -4T0 > name.tar.xz # FreeBSD
  

  On FreeBSD, the syntax is stricter; we have to use - as standard input/output to direct input/output. The option -T0 is for multithreading; it uses all possible cores as the number of job workers. -4 is compression level; the default level is 6 but most likely it takes significantly longer time.

• Place the cursor in front of a character and use M-x describe-char to check out the information about this character. For example ğ can be inserted into Emacs buffer by C-x 8 RET 11f.
• Enabling flyspell-mode mode in org-mode slows down the org-ac completion so much that the latter is almost useless. Use on demand spell check instead. For example, M-$ a word or M-x ispell-region a block of text.

• Switch to xetex from latex. Add the following to the *.tex (on a per file basis).

  %%% Local Variables: 
  %%% coding: utf-8
  %%% mode: latex
  %%% TeX-engine: xetex
  %%% TeX-master: t
  %%% End:
  

  The reason of this switch was I chanced upon the font used in A.S. Byatt’s novel Possession that I bought long time ago. To use the font with ease, we need to load fontspec package and add the following to the preamble of a *.tex file.

  ...
  
  % a very beautiful font as I saw in A.S. Byatt's Possession
  \setmainfont{IMFePIrm29C.otf}[   % see details in fontspec doc 
  Path          = /path/to/fonts/, % the ending forward slash is required
  % Extension     = ttf,
  ItalicFont    = IMFePIit29C.otf,
  SmallCapsFont = IMFePIsc29P.ttf,
  StylisticSet  = 2,               % Alternate below defaults to 0
  WordSpace     = 1.2,
  Ligatures     = Rare,
  Style         = Alternate,       % access alternate glyph
  ]
  
  ...
  

  The font’s full name is IM Fell Type after John Fell. It is hosted at Google Fonts and the original author’s website.

• Add fonts to Emacs. See here in the official documentation. For installing fonts without Admin Access on Windows, see here for a solution using Portable Apps.
• Common html characters for typography

• Overwrite the default postamble format when exporting *.html’s.

  • First access the default format: C-h v org-html-postamble-format.
  • Then hardcode the customized format with #+BIND:; we need to change variable org-html-postamble from auto to t.

  #+BIND: org-html-postamble t
  #+BIND: org-html-postamble-format (("en" "<p class=\"author\">Author: %a</p>\n<p class=\"date\">Last updated: %d</p>")))
  #+DATE: <2017-10-12 Thu 12:31>
  

  • Finally, in order to allow #+BIND: to work, set org-export-allow-bind-keywords as non-nil.

  # Local Variables:
  # org-export-allow-bind-keywords: t
  # End:
  

• Install FastX on https://chipnotized.org server. The latest emacs 25.x is nowhere to be found on any campus Linux box (on some workstations, I saw even emacs 21.x), forcing me to make this move :(. See here for server side installation without root privilege.

  As an alternative, use X11 forwarding instead. (Testing)

  • On server side, add AllowX11Forwarding yes to /etc/ssh/sshd_config.
  • On client side, add ForwardX11 yes to /etc/ssh/ssh_config.

  A major problem here was I need to work on the server, but Nextcloud client wouldn’t resolve local IP address. An https address is a required field the first time Nextcloud client is initiated. The fix is quite an easy one but it took me a while to figure out: go to nginx configuration file for nextcloud in /etc/nginx/sites-available and add 127.0.0.1 after the resolvable domain name in the server_name directive. Then use 127.0.0.1 as the server address when Nextcloud client prompts for an input. The server has to use the localhost to see itself.

  Update (Nov 30, 2017): Switched to NoMachine NX free version after struggling with licensing issue with FastX 2. U of I Webstore license seems to require all servers to be behind university firewall. Installation of NX can be found here. Also, it is not true that in order to use NX client, we have to use sudo install. Simply unpack nxplayer somewhere you have the write privilege, then you are good to go.

• Change SageMath terminal colors. Add

  %colors Linux
  

  to $HOME/.sage/init.sage or run it from sage prompt.

• Precompiled binaries from sagemath.org won’t start on an Apollo Lake CPU powered machine; it won’t plot() on ThinkPad X41/T running 32bit Linux either.

  Building Sage from scratch. Follow developer’s guide to get a copy of the latest code. Stable release didn’t build through in either case. (At some point, it complained that openblas couldn’t detect CPU architecture.) Set architecture manually for Apollo Lake Atom CPU,

  $OPENBLAS_CONFIGURE="TARGET=NEHALEM" MAKE='make -j8' SAGE_NUM_THREADS=8 SAGE_INSTALL_GCC=yes make 
  

• Dell UZ2315H RGB Linux setting RGB (97, 99, 94), Win setting RGB (100, 94, 85).

This is a how-to for upgrading nextcloud server. See the manual upgrade for more.

• From within the server, first make a copy of current working /var/www/nextcloud

  $cd /var/www
  $sudo cp -rv nextcloud nextcloud.bak
  

• Then stop the web server

  $sudo service nginx stop
  

• Make a backup of working nextcloud directory

  $sudo mv -v nextcloud nextcloud.old
  

• Unpack downloaded the latest nextcloud tarball and unpack it then move it to /var/www. Verify the download with diff. (If md5sum agrees, then we see no standard output.)

  $cd ~/Downloads/
  $aria2c https://download.nextcloud.com/server/releases/nextcloud-13.0.4.tar.bz2
  $aria2c https://download.nextcloud.com/server/releases/nextcloud-13.0.4.tar.bz2.md5
  $md5sum nextcloud-13.0.4.tar.bz2 | diff -u nextcloud-13.0.4.tar.bz2.md5 -
  $cd nextcloud-latest/
  $tar xavf ../nextcloud-13.0.4.tar.bz2
  $cd /var/www/
  $sudo mv -v ~/Downloads/nextcloud-latest/nextcloud/ ./
  

• Change directory permissions using ~/nextcloud_dir_config_script to prepare nextcloud folder. This script will create necessary folders first then set correct ownerships.

  #!/bin/bash 
  ocpath='/var/www/nextcloud' 
  htuser='www-data' 
  htgroup='www-data' 
  rootuser='root' 
  
  printf "Creating possible missing Directories\n" 
  mkdir -p $ocpath/data 
  mkdir -p $ocpath/updater 
  
  printf "chmod Files and Directories\n" 
  find ${ocpath}/ -type f -print0 | xargs -0 chmod 0640 
  find ${ocpath}/ -type d -print0 | xargs -0 chmod 0750 
  
  printf "chown Directories\n" 
  chown -R ${rootuser}:${htgroup} ${ocpath}/ 
  chown -R ${htuser}:${htgroup} ${ocpath}/apps/ 
  chown -R ${htuser}:${htgroup} ${ocpath}/config/ 
  chown -R ${htuser}:${htgroup} ${ocpath}/data/ 
  chown -R ${htuser}:${htgroup} ${ocpath}/themes/ 
  chown -R ${htuser}:${htgroup} ${ocpath}/updater/ 
  
  chmod +x ${ocpath}/occ 
  
  printf "chmod/chown .htaccess\n"
  if [ -f ${ocpath}/.htaccess ]
   then
    chmod 0644 ${ocpath}/.htaccess
    chown ${rootuser}:${htgroup} ${ocpath}/.htaccess
  fi
  if [ -f ${ocpath}/data/.htaccess ]
   then
    chmod 0644 ${ocpath}/data/.htaccess
    chown ${rootuser}:${htgroup} ${ocpath}/data/.htaccess
  fi 
  

• Copy /var/www/nextcloud.old/config/config.php to /var/www/nextcloud/config/ (we need su -) and Copy /var/www/nextcloud.old/data/* to /var/www/nextcloud/data/ (also need su -).
• Change directory permissions using ~/nextcloud_dir_config_script again.

• Start web server

  $sudo service nginx start
  

• Upgrade nextcloud

  $cd /var/www
  $sudo -u www-data php ./nextcloud/occ upgrade
  

Wait and see if screen log shows successful upgrade.

• If a spinning disk was misidentified as a solid state device, run winsat formal -v. It will reassess computer hardware and according to hard drive performance it readjusts the how fast OS boots as well.
• Disable “Microsoft Compatibility Telemetry”. Run taskschd.msc. Find the folder Task Scheduler Library -> Microsoft -> Windows -> Application Experience. Right click on any of the tasks listed as Microsoft Compatibility Appraiser and choose Disable.

• Build Nextcloud client for Windows with the latest support of OpenSSL. Only supported on openSUSE. Use openSUSE Tumbleweed for latest tool chains.

  • Clone client_theming repo.

  ~>git clone https://github.com/nextcloud/client_theming.git
  ~>cd client_theming
  client_theming>git submodule update --init --recursive
  

  • If the previous step is not working, remove client folder and manually checkout ownCloud repo.

  client_theming>git clone --recursive https://github.com/owncloud/client.git
  

  • Check out the latest ownCloud client.

  client_theming>cd client/
  client_theming/client>git checkout v2.4.1
  

  • Start docker service and build docker image.

  client_theming>sudo service docker start
  client_theming/client>sudo docker build -t nextcloud-client-win32:2.4.1 client/admin/win/docker/
  

  • Build the client.

  client_theming>sudo docker run -v "$PWD:/home/user/" nextcloud-client-win32:2.4.1 /home/user/win/build.sh $(id -u)
  

  The resulting binary is in client_theming/build-win32/. To use the latest tool chains, modify /client/admin/win/docker if necessary.

  To clean the docker builds, see how to remove docker images, containers, volumes.

• Install microcode with VMware CPU Microcode Update Driver.
• Manually set Solarized theme for Win PowerShell. Click the upper left corner of PowerShell and follow Properties – Colors – Screen Backgroud. Set RGB colors as (0, 43, 54). Set font PragmataPro, size 18. Cygwin terminal can also use PragmataPro font; set font size 12.
• The more recent versions of PrgmataPro font do not look as nice as before in Emacs, plus it is not free. Use Hack instead.

• Remove recent archive names in WinRAR. Open regedit and find

  Computer\HKEY_CURRENT_USER\Software\WinRAR\DialogEditHistory\ArcName
  

  Recent histories are numbered on the right. Check each by hand by double clicking.

• After cloning current SSD to an image SSD with EZ Gig IV Cloning Software, Windows won’t boot.

  Your PC needs to be repaired. A required device isn't connected or can't be accessed.
  

  The fix:

  • Create a recovery drive with Win 10 built-in tool with a USB drive.
  • Depending on whether the machine is a BIOS Legacy boot or a UEFI boot, change BIOS settings and boot into corresponding mode using the USB recovery drive. For example, boot the USB repair drive into BIOS mode to fix a BIOS boot machine or boot into UEFI mode to fix a UEFI boot machine.
  • Follow Repair your computer – Advanced options – Command prompt to enter cmd.exe Windows prompt.
  • Rebuild BCD.

  bootrec /fixmbr
  bootrec /fixboot
  bootrec /rebuildbcd
  

• Install portable GnuPG. Assume we copied all necessary binaries to a folder GnuPG where gpgconf.exe resides. Create an empty file gpgconf.ctl under the same directory then GnuPG will be GNUPGHOME, all other methods of setting GNUPGHOME will be ignored.
• Disable cortana and telemetry feature of Windows 10.
  • Fire up regedit.exe and find HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\DataCollection. Under that folder create AllowTelemetry DWord entry and set it as 0.
  • Next, disable Connected User Experiences and Telemetry and dmwappushsvc in servies.msc.
  • Go to settings, scroll to the bottom, find App Diagnostics and turn it off; one item above is Background apps, turn it off as well.
• Color profile folder location. On Win 7, it is C:\Windows\System32\spool\drivers\color.