Wednesday, August 22, 2012

Bypass proxy script for Pacman

A few months back I had written a small bash script to bypass the download limit (14MB) imposed by our proxy server. What it essentially does is, downloads files chunk-wise; for instance, in my case, it'd download 14MB chunks at a time and appends it to the file... all on-the-fly! It served me pretty well all these days. Moreover, I also wanted to use it as my Pacman's XferCommand to stay up-to-date.

Though the original version works, it needs a few tweaks. For bigger files, output doesn't look streamlined. Like this...


I removed all unwanted code like i) filename guessing from URLs and ii) output directory validations. And beautified/simplified it's output to look like the below screenshot. To use it with pacman, go to /etc/pacman.conf and add a XferCommand like shown in the top pane of the below screenshot. That's it, now it works well with pacman too...





Here's the new code... It's also on github

#!/bin/bash
#
# Vikas Reddy @
# http://vikas-reddy.blogspot.in/2012/04/bypass-proxy-servers-file-size-download.html
#
#

# Erase the current line in stdout
erase_line() {
    echo -ne '\r\033[K'
}

# Asynchronously (as a different process) display the filesize continously
# (once per second).
# * Contains an infinite loop; runs as long as this script is active
# * Takes one argument, the total filepath
async_display_size() {
    PARENT_PID=$BASHPID
    {
        # Run until this script ends
        until [[ -z "$(ps x | grep -E "^\s*$PARENT_PID")" ]]; do
            # Redraw the `du` line every second
            erase_line
            echo -n "$(du -sh "$1") "
            sleep 1
        done
    }&
    updater_pid="$!"
}

# Defaults
fsize_limit=$((14*1024*1024)) # 14MB
user_agent="Firefox/20.0"


# Command-line options
while getopts 'f:d:u:y' opt "$@"; do
    case "$opt" in
        f) filepath="$OPTARG"    ;;
        u) user_agent="$OPTARG"  ;;
    esac
done
shift $((OPTIND - 1))

# Exit if no URL or filepath argument is provided
if [[ $# -eq 0 ]] || [[ -z "$filepath" ]]; then
    exit
fi

# Only one argument, please!
url="$1"

# Create/truncate the output file
truncate --size 0 "$filepath"


# Asynchronously (as a different process) start displaying the filesize
# even before the download is started!
async_display_size "$filepath"

# infinite loop, until the file is fully downloaded
for (( i=1; 1; i++ )); do

    # setting the range
    [ $i -eq 1 ] && start=0 || start=$(( $fsize_limit * ($i - 1) + 1))
    stop=$(( $fsize_limit * i ))

    # downloading
    curl --fail \
         --location \
         --user-agent "$user_agent" \
         --range "$start"-"$stop" \
         "$url" >> "$filepath" 2> /dev/null; # No progress bars and error msgs, please!

    # catching the exit status
    exit_status="$?"

    if [[ $exit_status -eq 22 ]] || [[ $exit_status -eq 36 ]]; then
        # Download finished
        erase_line
        echo "$(du -sh "$filepath")... done!"
        break
    elif [[ $exit_status -gt 0 ]]; then
        # Unknown exit status! Something has gone wrong
        erase_line
        echo "Unknown exit status: $exit_status. Aborting..."
        break
    fi

done


This can also be used as a standalone script to download files normally. Just that you need to mention the full filepath of where you want to download. Like this...

./pacman-curl.sh -f ~/downloads/video.mp4 'http://the-whole-url/'




1 comment:

لاعبه الجمباز said...
This comment has been removed by a blog administrator.