Monday, September 28, 2009

ShellSink adds inline tagging

Release 0.2.2 adds a new feature. You can add tags to a command at the same time as you execute it. This is achieved by using inline comments. Here is an example:
cat /proc/cpuinfo #sysadmin:diagnostic
That command will show up in your shellsink history with the tags sysadmin and diagnostic. This nifty idea was given to me by Jesse Hallett at my shellsink presentation at the open source bridge conference in portland this summer. Thanks Jesse.

Saturday, May 2, 2009

Filtering What Goes Into Your History


When I tell people about ShellSink, many of them say, "hey, that's a great idea!" Then the little sysadmin perched on their shoulder crosses her arms and shakes her head. "Tsk, tsk. Storing your shell history on a remote server is not secure!", she says. This is absolutely true. Not only is your bash history passed via an insecure Http connection, but then it is stored unencrypted on Google's servers where a rogue Google employee, someone who hacked Google, or a rogue ShellSink admin could access them. Don't use shellsink on a system that needs to be ultimately secure!

However, Shellsink is safe to use. If you use Google for email, calendar, documents, etc. then you already trust them with information that is probably more sensitive than your shell history. And if you don't trust me you can download the ShellSink source code and spin up your own server using Google's AppEngine. In any case you should still take some basic precautions. Most importantly, never use inline passwords!! ShellSink will send anything your bash history captures.

Here are several ways to safeguard yourself against sending sensitive information to ShellSink. One technique is to add the following to your .bashrc:
HISTCONTROL=ignorespace;
Now any command you precede with a space will NOT be captured by the history, and thus will not be sent to the ShellSink server. Another technique is to add the following to your .bashrc:
HISTIGNORE="netstat*:sqlplus*"
In this example any command that starts with netstat or sqlplus will automatically be ignored by the shell history and ShellSink. Both these techniques give you different granularity control over what gets sent to ShellSink. The first technique works on a per command basis and the second covers all commands.

Finally, there is the nuclear option:
shellsink-client -d
This completely disables ShellSink for all terminals. I don't really have much use for this one, but when you start to do something that you are ultra paranoid about, this is one way to be sure nothing is leaving your computer via ShellSink. To re-enable ShellSink simply do this:
shellsink-client -e

ShellSink at the Open Source Bridge Conference


I'm going to be in Portland for the Open Source Bridge Conference (OSB) to talk about ShellSink. The abstract I submitted is overly ambitious as I promised to cover shellsink usage, implementation as well as distribution via Launchpad. I've decided to stick to usage and implementation of shellsink as I am giving a presentation entirely dedicated to distributing software via launchpad at OSCON this year. Of course, OSB has the totally awesome 24hour hacker lounge as well as an entire day dedicated to uncoference antics and open space shennanigans, so if anyone feels shortchanged I'd be happy to spend some time talking about launchpad (or whatever) in any of those situations. Please drop a line if you are going to be at OSCON or OSB this year.

Thursday, February 26, 2009

Shellsink Nearing Free Quota

Shellsink has been a pet project of my for the last few months, and it has been fun to put something new out there just to see how people react. My main goal of course was to build something useful. What I didn't anticipate was that I would have to begin paying for hosting so soon. Unfortunately Google has rolled out the non free portion of app engine, and they have announced resulting changes to the service. I had hoped that ad revenue on the shellsink site would pay for whatever app engine hosting costs were incurred, but since 99% of use is at the command line, actually very very few ad impressions are viewed on the site.

The short of this is that very soon shellsink will hit the data storage quota. 1GB of commands, tags, and annotations will have been logged and indexed on the shellsink site. I an trying to get Google to lift the quota for my application. But I have little hope unless one of the Googlers using the service decides to intervene. I expect that late next week shellsink.com will hit the quota and stop accepting commands. BUMMER!

The good news is that all data you have logged there will remain accessable, but if you are like me, that isn't much consolation. So the other good news is that all the source code for the server is going to be in the shellsink Launchpad bazaar repo, so you can keep using shellsink by deploying to your own app engine account. So, it looks like without a deux ex-machina that shellsink.com will have to morph into a non-centralized system.

UPDATE: There is a thread in the google group about this.

Sunday, February 8, 2009

Using shellsink with ZSH is easy


You don't have to use bash to use shellsink. Shellsink will work with any shell that supports the following features:
  • Concept of a command that always executes before your command is run
  • Concept of a history file
  • Concept of appending to the history file each time you run a command
Z-Shell is another great shell and when configured correctly it supports all these things. Here is a set of instructions to make shellsink work for zsh.
  1. Grab the sample zshrc
  2. Concatenate that to ~/.zshrc
  3. make a symlink from .bash_history to .zhistory like this:
    ln -s ~/.zhistory ~/.bash_history
  4. Restart your shell

Easy, right? Here is a list of my References:
  • http://en.gentoo-wiki.com/wiki/Zsh#Shell_History
  • man zshmisc
  • man zsh

Monday, January 19, 2009

ShellSink Server Upgrade

On the same day that our government gets an upgrade, ShellSink gets an upgrade. Specifically, a new version of the server is live! The changes are all performance related and it enables me to bring the client behavior into line with the way google has designed app engine to scale. That is, to make lots of very simple requests instead of fewer complex requests.

Unfortunately, this version is not backwards compatible with older versions of the client. You must upgrade your client! Why would I make such a drastic change? Simple, nobody is using shell_sink but me! That means I'm sure 100% of my users will upgrade. It doesn't explain why I need a blog entry to tell myself about it.

Wednesday, January 7, 2009

How to use tags in the sink.


Tags can be useful in many different ways. I thought I would share some of the ways I use tags.

  1. Using tags to say what computer has logged the command: I set the SHELL_SINK_TAGS environment variable in my bash_profile to indicate what machine logged a particular command like this:
    SHELL_SINK_TAGS=myhost
    This is particularly useful because I let several machines log their commands to the same shell-sink account, and if I didn't do that I wouldn't be able to remember which machine logged what.
  2. Using tags to group commands: Often times when I am working on a particular project I want to indicate that the commands I am issueing are related to that project. This makes it easy to remember the context of commands in my history and helps me find things later. Let's say I have a project called "gloop". When I set down to work on gloop I'll set my tags environment variable to indicate that as follows:
    export SHELL_SINK_TAGS=$SHELL_SINK_TAGS:gloop
    Now any commands issue in the terminal I am working in will get tagged with gloop. When I want to stop tagging with the gloop tag I simply issue the following command
    source ~/.bash_profile
    . If your SHELL_SINK_TAGS variable is defined in a different file than mine you will need to source the appropriate file for your environment. On ubuntu that file is ~/.bashrc
  3. Using tags to control your RSS feed: If you go to the preferences page of your shellsink account you will see that you can turn on an rss feed. You can also filter your rss feed by any tag, so if you only want people to be able to see certain commands in the feed you can choose the appropriate tag from the droplist. As soon as you select something in the droplist only commands with that tag will appear in your RSS feed.
  4. Using tags to pull: The shellsink-client application normally just pushes your commands out to the server, but it can also pull commands from the server and put them into your history file. If you issue
    shellsink-client -pt gloop
    . The most recent 20 commands tagged with gloop will be pulled into your bash history file. This is especially useful for making all the commands for a particular task immediately available in your history file.
These are just a few of the ways you can use tags in shellsink. Some things to remember while using tags. Currently you can only add a maximum of four tags at a time. When adding multiple tags from the client make sure you use a colon delimited list of tags. When adding multiple tags from the web application use a comma delimited list of tags. Enjoy, and please leave your own recipes for working with tags here in the comments.

Photo Credit: sarah parrott