Linux tips

This for the most part isn't splunk-specific, but if you do any amount of administration on the linux command line, you might find it helpful.

.bashrc

A better prompt

In your .bashrc:

PS1='\t \[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '

This does a number of things:

• adds some colours: green for username and hostname, blue for current working directory. Sometimes when a command outputs a lot of text, it can be difficult to spot where it started. This makes it much easier

• the full current working directory is shown, instead of just the last bit

• the format of the prompt is compatible with rsync or scp: username@hostname:/full/path. This makes copy paste your friend sometimes

• adds a timestamp: this doesn't hurt and is useful occasionally and may give a hint as to how long something took or when it finished

History

The history of past commands is incredibly useful. You want to make sure you don't lose it.

In your .bashrc:

# 1000 times more commands in history than the default
HISTSIZE=1000000
# allow history filesize to be 2000 times bigger than the default
HISTFILESIZE=2000000
# Avoid duplicates in history
export HISTCONTROL=ignoredups:erasedups
# When the shell exits, append to the history file instead of overwriting it. This is important when more than one shell is open at a time.
shopt -s histappend
# Updates the history file after each command
export PROMPT_COMMAND="history -a; $PROMPT_COMMAND"

Other

Can't hurt, and if you've ever been bit you'll be glad:

# Must press ctrl-D twice to exit (instead of once!)
export IGNOREEOF=1

Aliases

You might find some of this useful for your .bashrc:

# lazy
alias l=ls

# indulge bad habit taken up in my DOS days
alias cd..="cd .."

# Make cp and mv safer
alias cp="cp -i"
alias mv="mv -i"

# useful to figure out where disk space goes
alias dud="du -h --max-depth=1"
alias duds="du --max-depth=1 | sort -n"

# colours!
alias grep="grep --color

Splunk-specific aliases

Again in your .bashrc:

alias tailsplunkdlog="tail -f /opt/splunk/var/log/splunk/splunkd.log"
alias cdetc="cd /opt/splunk/etc/"
alias vimpropsandtransforms="vim -p default/props.conf default/transforms.conf local/props.conf local/transforms.conf"
alias vimeventtypes="vim -p default/eventtypes.conf default/tags.conf local/eventtypes.conf local/tags.conf"
alias grepsplunk="grep --exclude-dir \"*.index\" --exclude-dir \"default.old*\" --exclude-dir replication"
alias serverclass="vim /opt/splunk/etc/system/local/serverclass.conf"

export PATH=$PATH:/opt/splunk/bin/

That last bit allows you to enter commands like "splunk status", "splunk restart" etc, without having to change dir or specify full paths.

.inputrc

history search

This is one of my favourite tips! It makes it easier to make the most of your past command history.

First a bit of context about how you can use bash history without this change:

• use the UP arrow (and then UP/DOWN as desired) to bring back the last command(s) in the history. Commands are not filtered, so you may need to press the up arrow a lot to get to what you want

• use ctrl-R and start typing to show the last command that matches what you're typing. Press ctrl-R again to go to the next match without changing the pattern

Both are very useful, but the following .inputrc lines take it a step further:

# Up/down arrow to search history for commands starting the same
# (Old behaviour still available with ctrl+P and ctrl-N)
"\e[A": history-search-backward
"\e[B": history-search-forward

If you start typing a command, e.g. "spl", and then press the up arrow, bash will conjure the last command in your history that started the same, e.g. "splunk status". With more hit on up/down arrows you can navigate all previous commands that start the same. If you start with an empty line, the up/down arrows behaviour is the same as before.

The new behaviour might trip you up. If it does, just hit ctrl-C to start with an empty line and it'll go back to normal. But if you know the command you're after, it'll bring it forth very quickly.

navigation helps

These are useful when double tapping the TAB key, to see what files are matching what you started typing:

# Ignore hidden files when matching from scratch
set match-hidden-files off
# show what completion choices are (directories, executables or other)
set visible-stats on

And this just makes editing your current line easier:

# ctrl-left/right jumps whole words
"\eOC": forward-word
"\eOD": backward-word

vim

.vimrc

Vim is a very powerful editor if you take the time to learn it. If you want to learn vim and like Zelda, I recommend Vim Adventure. In any case, here is some basic config for it:

" make vimdiff more readable
set diffopt+=iwhite

" allow arrow keys to go past the end/beginning of lines
set ww=<,h,l,>

" make search case insensitive, unless the pattern being searched is not entirely lowercase
set ignorecase
set smartcase

" no tabs, and consistent indent
set smartindent
set expandtab
set shiftwidth=2
set tabstop=2

" disable beeps
set noeb vb t_vb=

" shortcut to go in/out of paste mode
set pastetoggle=<F8>

Splunk syntax highlighting

This is a must if you're using vim to edit any splunk file. It literally spots typos for you.

https://github.com/yorokobi/vim-splunk

bash tips

completion

Make sure you make the most of bash's completion features:

• use TAB to auto complete paths and filenames

• use double TAB to show options for completion when there's more than one

alt dot

This is one of my favourite tip ever. Sadly it doesn't work on apple's terminal. :(

Use "alt dot" or alt-. to bring up the last word of the last command. If you keep pressing it, it'll bring up the last word of the command before that, and the one before that, etc. This is incredibly useful!

Here's an example:

mkdir averylongdirectoryname
cd averylongdirectoryname

You can achieve the above by typing:

mkdir averylongdirectoryname<ENTER>
cd <ALT-.>

Of course you could also start typing the path name and use <TAB> but <ALT-.> works better if you have multiple files with similar names that TAB would trip on and it works with things that are not files. For instance in the following exampel it would save you a lot of typing and/or some copy-pasting:

rsync -az file1 myuser@myhostname:some/very/long/path
rsync -az file2 myuser@myhostname:some/very/long/path

script to colour the output of diff

GNU diff is incredibly useful, but it can be difficult to read the output. I often pipe the output to a simple perl script I wrote to colour the output:

diff -U3 BEFORE AFTER | colour-diff

I don't even type the whole "colour-diff", usually "colo" and <TAB> is all that's needed.

Shove this in ~/bin/colour-diff and don't forget to do "chmod +x ~/bin/colour-diff"

colour-diff

.txt

Download TXT • 3KB

grep VS highlight

Grep is incredibly useful, and you should learn how to use it.

By default grep only shows you the lines that match what you are looking for. You can mess with context options, but I found it useful to write a simple perl script that doesn't filter any thing but uses colour to highlight any matches. E.g.:

diff -q -r before/ after/ | highlight 'Only in'

Shove this in ~/bin/highlight and don't forget to do "chmod +x ~/bin/highlight"

highlight

.txt

Download TXT • 2KB