April 13, 2023

How to Navigate this Slide Show?


  • This ioslides presentation contains scrollable slides.
  • Which slides are scrollable, is indicated by a tag at the bottom of the corresponding slides stating:

[ Scroll down to continue ]

  • The following single character keyboard shortcuts enable alternate display modes of ioslides:
    • f: enable fullscreen mode
    • w: toggle widescreen mode
    • o: enable overview mode
    • h: enable code highlight mode
  • Pressing Esc exits all of these modes. Additional details can be found here.

Outline

  • Background
  • Neovim-based IDE for R
  • Conclusions
  • References

R Language

About

  • Statistical environment and programming language (CRAN) widely used in academia and data science.
  • Free and runs on all common operating systems
  • Large ecosystem of extension packages, e.g. Bioconductor and CRAN

Working environments

RStudio Server for Web-based HPCC Access

  • Integrated development environment (IDE) for R. RStudio local GUI and RStudio Server is web-based.
  • User access to RStudio Server on HPCC

Some useful RStudio shortcuts:

  • Ctrl+Enter: send code to R console
  • Ctrl+Alt+Enter: send code to terminal
  • Ctrl+Shift+C: comment/uncomment
  • Ctrl+1/2: switch window focus

Nvim-R-Tmux: Terminal-based R Environment

Advantages of Command-line UI

  • Knowledge of command-line interface is essential for working on a computer cluster efficiently
  • Main advantage: language agnostic approach that works with most computer languages
  • Users of Emacs may want to consider using ESS instead

Outline

  • Background
  • Neovim-based IDE for R
  • Conclusions
  • References

Introduction to Nvim-R-Tmux

  • Note: the content on the following slides is also available in this tutorial section here.
  • The examples are assuming that users are logged into their HPCC cluster account, or are working on a system where vim/nvim is accessible via a terminal.
  • The following introduces Nvim-R combined with Tmux.
  • Similar instructions are available in HPCC’s Nvim-R-Tmux tutorial here.
  • Note: Nvim-R and Tmux are two separate tools that can be used independently or in combination, and both are useful for remote terminal work.
  • For simplicity, some of the following examples use Nvim-R without Tmux. Once users know the basics of both then it is trivial to combine them as needed.

Vim/Nvim Basics

The following opens a file (here myfile.txt) with nvim (or vim). This can be a new file or an existing one.

nvim myfile.txt # for neovim (or 'vim myfile.txt' for vim)

Modes

In Vim/Nvim there are three main modes: normal, insert and command mode. The most important commands for switching between the three modes are:

  • i: switches from the normal to the insert mode. The latter is used for typing.
  • Esc: switches from the insert mode back to the normal mode.
  • :: starts the command mode (from normal mode) at the bottom of the terminal window.

The cursor is moved with the arrow keys. In Nvim one can also enable mouse-based movements of the cursor. Fn Up/Down allows to page. In the following, all commands starting with : need to be typed in the command mode. All other commands are typed in the normal mode after pushing the Esc key.

[ Scroll down to continue ]



Important modifier keys to control Vim/Nvim

  • :w: saves changes to file. If in editing mode, Esc needs to be pressed first.
  • :q: quits file that has not been changed; use q! to quit file without saving changes.
  • :wq: saves and quits file

Useful resources for learning Vim/Nvim

Quick Nvim-R Demo (details on next slides)

To try out the following instructions, users want to log into their HPCC account via ssh, and then preferentially connect to a node by initializing an interactive srun session. The latter mimics the best practices for a real workflow but is not necessary for this basic exercise. If Nvim-R is not installed in a user’s account yet, then configure it with the Install_Nvim-R_Tmux command as outlined here.

srun --x11 --partition=short --mem=2gb --cpus-per-task 4 --ntasks 1 --time 1:00:00 --pty bash -l
  • Under --partition it is important to assign the name of a partition a user has access to
    • Most users have access to: short, batch, intel and highmem
    • Users of labs owning computer nodes also can access: <pi_name>lab
  • For more details on argument settings for srun, see here

Download R_for_HPC_demo.R or nvimr_demo.R file to you HPCC account as follows.

wget https://raw.githubusercontent.com/tgirke/GEN242/main/static/custom/slides/R_for_HPC/demo_files/R_for_HPC_demo.R
wget https://raw.githubusercontent.com/tgirke/GEN242/main/static/custom/slides/R_for_HPC/demo_files/nvimr_demo.R

[ Scroll down to continue ]



Open R_for_HPC_demo.R or nvim_demo.R with nvim. The content of R_for_HPC_demo.R file is shown in the following code block. Next, initialize a Nvim-connected R session with \rf, and then execute the code by pressing the space bar on your keyboard.

library(tidyverse)                                                                                                                                                            
write_tsv(iris, "iris.txt") # Creates sample file                                                                                                                             
read_tsv("iris.txt") %>% # Import with read_tbv from readr package                                                                                                            
    as_tibble() %>%                                                                                                                                                           
    group_by(Species) %>%                                                                                                                                                     
    summarize_all(mean) %>%                                                                                                                                                   
    reshape2::melt(id.vars=c("Species"), variable.name = "Samples", value.name="Values") %>%                                                                                  
    ggplot(aes(Samples, Values, fill = Species)) +                                                                                                                            
    geom_bar(position="dodge", stat="identity")

[ Scroll down to continue ]





If X11 is enabled in a user session then the above code will generate the following bar plot in a separate graphics window.

Tmux for Managing Terminal Sessions

What is Tmux?

  • Tmux is a virtual terminal multiplexer providing re-attachable terminal sessions
  • Advantage: work in a terminal session cannot get lost due to internet disruptions or even when switching computers
  • Combined with the Nvim-r plugin it provides a flexible working environment for R
  • Users can send code from a script to the R console or command-line.
  • On HPCC both Nvim-R and Tmux are pre-configured and easy to install





Nvim-R-Tmux Configuration in HPCC User Accounts

Skip these steps if Nvim-R-Tmux is already configured in your account. Or follow the detailed instructions to install Nvim-R-Tmux from scratch on your own system (e.g. laptop or computer).

1. Log in to your user account on HPCC and execute on the command-line:

Install_Nvim-R_Tmux 

2. To enable the nvim-R-tmux environment, log out and in again.

3. Follow usage instructions of next section.

Typical Usage Workflow for Nvim-R-Tmux

1. Start tmux session from login node (not compute node!)

Running Nvim from tmux provides reattachment functionality. Skip this step if this is not required.

tmux # starts a new tmux session 
tmux a # attaches to an existing or preconfigured session 

2. Open nvim-connected R session

Open a *.R or *.Rmd file with nvim and initialize a connected R session with \rf. Note, the resulting split window among Nvim and R behaves like a split viewport in nvim or vim meaning the usage of Ctrl-w w followed by i and Esc is important for session navigation.

nvim myscript.R # or *.Rmd file

3. Send R code from nvim to the R pane

Single lines of code can be sent from nvim to the R console by pressing the space bar. To send several lines at once, one can select them in nvim’s visual mode and then hit the space bar.

[ Scroll down to continue ]



  • Please note, the default command for sending code lines in the nvim-r-plugin is \l. This key binding has been remapped in the provided .config/nvim/init.vim file to the space bar. Most other key bindings (shortcuts) still start with the \ as LocalLeader, e.g. \rh opens the help for a function/object where the cursor is located in nvim. More details on this are given on the next slide(s).
  • The most comprehensive manual on this is the official Nvim-R documentation here.

Keybindings to Control Environment

Important keybindings for nvim

  • \rf: opens vim-connected R session. If you do this the first time in your user account, you might be asked to create an R directory under ~/. If so approve this action by pressing y.
  • spacebar: sends code from vim to R; here remapped in init.vim from default \l
  • :split or :vsplit: splits viewport (similar to pane split in tmux)
  • gz: maximizes size of viewport in normal mode (similar to Tmux’s Ctrl-a z zoom utility)
  • Ctrl-w w: jumps cursor to R viewport and back; toggle between insert (i) and command (Esc) mode is required for navigation and controlling the environment.
  • Ctrl-w r: swaps viewports
  • Ctrl-w =: resizes splits to equal size
  • :resize <+5 or -5>: resizes height by specified value
  • :set mouse=a: enables mouse support and : set mouse-=a disables it


[ Scroll down to continue ]





  • :vertical resize <+5 or -5>: resizes width by specified value
  • Ctrl-w H or Ctrl-w K: toggles between horizontal/vertical splits
  • Ctrl-spacebar: omni completion for R objects/functions when nvim is in insert mode. Note, this has been remapped in init.vim from difficult to type default Ctrl-x Ctrl-o.
  • :h nvim-R: opens nvim-R’s user manual; navigation works the same as for any Vim/Nvim help document
  • :Rhelp fct_name: opens help for a function from nvim’s command mode with text completion support
  • Ctrl-s and Ctrl-x: freezes/unfreezes vim (some systems)

Important keybindings for tmux

Pane-level commands

  • Ctrl-a %: splits pane vertically
  • Ctrl-a ": splits pane horizontally
  • Ctrl-a o: jumps cursor to next pane
  • Ctrl-a Ctrl-o: swaps panes
  • Ctrl-a <space bar>: rotates pane arrangement
  • Ctrl-a Alt <left or right>: resizes to left or right
  • Ctrl-a Esc <up or down>: resizes to left or right

Window-level comands

  • Ctrl-a n: switches to next tmux window
  • Ctrl-a Ctrl-a: switches to previous tmux window
  • Ctrl-a c: creates a new tmux window
  • Ctrl-a 1: switches to specific tmux window selected by number

Session-level comands

  • Ctrl-a d: detaches from current session
  • Ctrl-a s: switch between available tmux sessions
  • $ tmux new -s <name>: starts new session with a specific name
  • $ tmux ls: lists available tmux session(s)
  • $ tmux attach -t <id>: attaches to specific tmux session
  • $ tmux attach: reattaches to session
  • $ tmux kill-session -t <id>: kills a specific tmux session
  • Ctrl-a : kill-session: kills a session from tmux command mode

Use Same Environment for Other Languages

Basics

For languages other than R one can use the vimcmdline plugin for nvim (or vim). Supported languages include Bash, Python, Golang, Haskell, JavaScript, Julia, Jupyter, Lisp, Macaulay2, Matlab, Prolog, Ruby, and Sage. The nvim terminal also colorizes the output, as in the screenshot below, where different colors are used for general output, positive and negative numbers, and the prompt line.

vimcmdline

Install

To install it, one needs to copy from the vimcmdline repository the directories ftplugin, plugin and syntax and their files to ~/.config/nvim/. For user accounts of UCR’s HPCC, the above install script Install_Nvim-R_Tmux includes the install of vimcmdline (since 09-Jun-18).

Usage

The usage of vimcmdline is very similar to nvim-R. To start a connected terminal session, one opens with nvim a code file with the extension of a given language (e.g. *.sh for Bash or *.py for Python), while the corresponding interactive interpreter session is initiated by pressing the key sequence \s (corresponds to \rf under nvim-R). Subsequently, code lines can be sent with the space bar. More details are available here.

Selecting R Versions on HPCC

  • Like many other software tools, R versions are managed under HPCC’s module system.
  • To use a specific R version in Nvim-R, one simply loads it prior to starting Nvim. Instructions for enabling additional R version toggle options are provided here.

Which R versions are available can be listed with the following command.

module avail R

The version labeled default is used by default. A specific R version can be loaded as follows.

module load R/4.1.2

Check which modules (including R) are loaded in a user’s environment.

module list

Outline

  • Background
  • Neovim-based IDE for R
  • Conclusions
  • References

Conclusions

Nvim-R-Tmux

  • Steeper learning curve than GUI-based IDEs, including RStudio or Jupyter Notebooks
  • However, it is much more
    • powerful, flexible, robust and language agnostic solution for working on remote systems
    • very useful for parallelization tasks on clusters in combination with BatchTools (Bischl et al. 2015)
    • time learning it is well invested, especially for students and researchers with complex data analysis and programming needs

Outline

  • Background
  • Neovim-based IDE for R
  • Conclusions
  • References

References

Bischl, Bernd, Michel Lang, Olaf Mersmann, Jörg Rahnenführer, and Claus Weihs. 2015. “BatchJobs and BatchExperiments: Abstraction Mechanisms for Using R in Batch Environments.” Journal of Statistical Software. http://www.jstatsoft.org/v64/i11/.