TIME NOW
World current time now,
CALENDAR
Calendar monthly, yearly
login CONVERT LENGTH
login CONVERT TEMPERATURE
login DICTIONARIES, LISTS
login SCIENCE EDUCATION RELIGION
login WORK CALCULATOR
login CALCULATE LIFE

Linux commands, scripts, examples, tutorial

Previous articlePage bottomNext article  ALL TOPICS

Linux. Simple wget http client bot

 folder='/_scripts/_cron_init/'

y=0
for i in `cat ${folder}filtered.final `; do
# cp "$i" "$i".bak ;

  export http_proxy="http://$i"

  wget -a ${folder}tmp/zaidimas.log --save-headers -t 1 -w 5 -T 5 --proxy=on -P ${folder}tmp http://o-ho-ho.com
 
  if [ $y -gt 20 ]
  then
  y=0
  fi
  y=` expr $y + 1 `
  # k=` expr $y \* 2 `
  k=2
  echo ` date ` $y $i $k
  echo ` date ` $y $i $k >> ${folder}wget_z.log
  sleep $k
done
 
Previous articlePage topNext article  ALL TOPICS


Guest:
Hello

How to get site structure? Answer is here:
 
wget --spider --recursive --no-verbose --output-file=wgetlog.txt Your_site


2013-06-25 15:59:24
Guest:
Hello,

To download full site:

wget \
     --recursive --no-parent --no-clobber \
     --page-requisites --html-extension --convert-links \
     --restrict-file-names=windows --domains your_site.org \
     your_site.org/tutorials/html


2013-06-25 04:25:47

 Use username: Guest, Anonymous, Programmer






QUOTES:
Short as life is, we make it still shorter by the careless waste of time.
Victor Hugo
Mental health increases as we pursue reality at all cost.
M. Scott Peck
Today's preparation determines tomorrow's achievement.
Anonymous