Saturday, October 29, 2005
mac osx disk commands
Wednesday, October 26, 2005
sed dos2unix {Scanned}
Sed dos2unix
[user@host] # sed s/.$//g unixfile > unixfile.sed
Unfortunately, this also removes characters that you may not want removed (e.g. the "T" in "CDT"):
Another option uses sed again, but strips the specific character instead of the last character on each line:
[user@host] # sed s/^M//g unixfile > unixfile.sed2
One very important item to understand about this command is that the "^M" (control character) is not generated by typing the "^" character, and then the "M" character from your keyboard. Instead, it is accomplished by typing Ctrl-V and then Ctrl-M (the Ctrl key and the V or M key are pressed simultaneously). Typing this sequence will produce the "^M" (control character), which allows sed to locate and process it as instructed.
The most desirable is running the dos2unix utility against the file:
[user@host] # dos2unix unixfile unixfile.dos2unix
####################################################################################################
Convert dos text files to unix, and vice versa:
dos2unix file.txtunix2dos file.txttr -d '\015' < win.txt > unix.txt # if you can't find dos2unixsed -e 's/$/\r/' < unix.txt > win.txt # if you can't find unix2dos |
####################################################################################################
With vi
Notice that some programs are not consistent in the way they insert the line breaks so you end up with some lines that have both a carrage return and a ^M and some lines that have a ^M and no carrage return (and so blend into one). There are two steps to clean this up.
1. replace all extraneous ^M:
:%s/^M$//g
BE SURE YOU MAKE the ^M USING "CTRL-V CTRL-M" NOT BY TYPING "CARROT M"! This expression will replace all the ^M's that have carriage returns after them with nothing. (The dollar ties the search to the end of a line)
2. replace all ^M's that need to have carriage returns:
:%s/^M/ /g
Once again: BE SURE YOU MAKE the ^M USING "CTRL-V CTRL-M" NOT BY TYPING "CARROT M"! This expression will replace all the ^M's that didn't have carriage returns after them with a carriage return.
It also works with
:%s/\r//g
I think using this command is easier.
:set ff=unix //to unix file
:set ff=dos //to windows file
Or
:set fileformat=dos
:set fileformat=unix with:
:%s/^M/\r/g
works perfectly !!!
####################################################################################################
Quick Script
#!/bin/bash
# To replace dos linebreaks for Unix compatibility
echo "This script will replace the ^M line breaks from dos."
echo -n "Enter filename without extension: "
read file
echo -n "Enter extension: "
read ext
sed 's/\r//' $file.$ext > $file2.$ext
cp -f $file2.$ext $file.$ext
rm -f $file2.$ext
This script is the same as before, just minus one step.
#!/bin/bash
# To replace dos linebreaks for Unix compatibility
echo "This script will replace the ^M line breaks from dos."
echo -n "Enter filename: "
read file
sed 's/\r//' $file > 2$file
cp -f 2$file $file
rm -f 2$file
Heres another little script
#!/bin/sh
FILE="$1"
# Use sed with the -i command line for inline interpreting.
sed -i '' "s/\r//g" $FILE
####################################################################################################
Just trim
From the UNIX shell: tr -d "\015" < inputfile > outputfile
E.g.: tr -d "\015" < dosformatted.txt > unixformatted.txt
####################################################################################################
Lets replace it with a new line!
sed "s/^M/\n/g" replaces the ^M with a linux newline.
the ^M is written Ctrl-V Ctrl-M, not with the carrot char.
####################################################################################################
to address the problem of ^M ( <ctrl>M ) characters in multiple files
the following single line SHell command will be helpful
for name in `ls *.dat` ; do sed 's/^M//' $name > ${name/\.dat/N\.dat} ; mv ${name/\.dat/N\.dat} $name ; done
####################################################################################################
Now in C
/* A program to strip control characters */
#include <stdio.h>
FILE *in,*tmp;
int main(int argc, char *argv[])
{
int index,count;
unsigned char byte;
printf("Hello! Are you ready to get rid of those nasty crlf?\n");
if(argc<2) {
printf("You need to specify an input file\n");
return 1;
}
if((tmp = fopen("tmp.tmp","wb"))==0){
printf("We could not open the temportary file called tmp.tmp\n");
return 2;
}
if((in = fopen(argv[1],"rb"))==0){
printf("We could not open the input file called %s\n",argv[1]);
return 3;
}
do{
count = fread(&byte,1,1,in);
if(count == 1){
if(byte!=0x0d) fwrite(&byte,1,1,tmp);
}
}while(count==1);
fclose(tmp);
fclose(in);
rename("tmp.tmp",argv[1]);
return 0;
}
####################################################################################################
Sed Again and Again…
# Under UNIX: convert DOS newlines (CR/LF) to Unix format
bash$ sed 's/.$//' file # assumes that all lines end with CR/LFbash$ sed 's/^M$// file # in bash/tcsh, press Ctrl-V then Ctrl-M # Under DOS: convert Unix newlines (LF) to DOS formatC:\> sed 's/$//' file # method 1C:\> sed -n p file # method 2 And trim one more time…
tr -d [^M] < inputfile > outputfile ####################################################################################################
Now in Perl
One Command Line
The simplest perl script is this one: perl -pi -e 's/\r\n/\n/;' *.java
This does the reverse: perl -pi -e 's/\n/\r\n/;' *.java
Two Lines
If you wish to be a little more complicated, you can do the same in two lines of perl. This enables you to simply name the file(s) you wish to convert on the command line. It would be used like so: dos2unix-2line *.java
Here is what dos2unix-2line it looks like:
#!/usr/bin/perl -pis/\r\n/\n/; Here is what unix2dos-2line it looks like:
#!/usr/bin/perl -pis/\n/\r\n/; More perl…
#!/bin/shperl -p -i -e 'BEGIN { print "Converting DOS to UNIX.\n" ; } END { print "Done.\n" ; } s/\r\n$/\n/' $* Thursday, October 20, 2005
smb mount
mkdir -p ~/smb_mnt/172.24.5.187/Users
sudo mount -t smbfs -o
uid=1000,gid=1000,username=krink,workgroup=singlefin.net //172.24.5.187/Users ~/smb_mnt/172.24.5.187/Users
Monday, October 17, 2005
rc.firewall dual nic gateway Linux
#!/bin/bash
#
# rc.firewall This shell script takes care of iptables
#
# chkconfig: 2345 99 99
# description: setup iptables dual homed as a gateway
# processname: rc.firewall
# config: /etc/init.d/rc.firewall
#
# karl@webmedianow.com
# Last Modified: Mon Oct 17 18:57:04 PDT 2005
#################################################
DATE=`/bin/date --iso`
EXT_DEV="eth0"
INT_DEV="eth1"
EXT_IP=`ifconfig $EXT_DEV |grep 'inet addr'| awk '{print $2}'|sed -e
"s/addr\://"`
INT_IP=`ifconfig $INT_DEV |grep 'inet addr'| awk '{print $2}'|sed -e
"s/addr\://"`
BCAST_EXT_DEV=`ifconfig | grep -A 1 $EXT_DEV | awk '/Bcast/ { print $3
}' | sed -e s/Bcast://`
BMASK_EXT_DEV=`ifconfig | grep -A 1 $EXT_DEV | awk '/Mask/ { print $4 }'
| sed -e s/Mask://`
BCAST_INT_DEV=`ifconfig | grep -A 1 $INT_DEV | awk '/Bcast/ { print $3
}' | sed -e s/Bcast://`
BMASK_INT_DEV=`ifconfig | grep -A 1 $INT_DEV | awk '/Mask/ { print $4 }'
| sed -e s/Mask://`
EXT_NET="$BCAST_EXT_DEV/$BMASK_EXT_DEV"
INT_NET="$BCAST_INT_DEV/$BMASK_INT_DEV"
#################################################
echo "wan side = $EXT_DEV: $EXT_IP Network: $EXT_NET"
echo "lan side = $INT_DEV: $INT_IP Network: $INT_NET"
#################################################
#Known Hosts
ROUTER="192.168.0.1"
WORKSTATION1="10.0.0.33"
OFFICE=""
#################################################
UNIVERSE="0.0.0.0"
PRIVPORTS="1024:65535"
PREROUTE22="$WORKSTATION1"
ALLOWPING="$INT_NET $ROUTER"
ALLOWDNS_INT_DEV="$INT_NET"
ALLOWSSH_INT_DEV="$INT_NET"
ALLOWDHCP_INT_DEV="$INT_NET"
#################################################
## Flush Rulesets and Zero out counter
flush() {
/sbin/iptables -F
/sbin/iptables -t nat -F
/sbin/iptables -t mangle -F
/sbin/iptables -X
/sbin/iptables -t nat -X
/sbin/iptables -t mangle -X
/sbin/iptables -Z
/sbin/iptables -t nat -F POSTROUTING
}
#################################################
## Define Variables in /proc/sys/net
proc() {
#echo "#Disabling IP Spoofing attacks."
echo "2" > /proc/sys/net/ipv4/conf/all/rp_filter
#echo "#Don't respond to broadcast pings (Smurf-Amplifier-Protection)"
echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
#echo "#Block source routing"
echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route
#echo "#Kill timestamps"
echo "0" > /proc/sys/net/ipv4/tcp_timestamps
#echo "#Enable SYN Cookies"
echo "1" > /proc/sys/net/ipv4/tcp_syncookies
#echo "#Kill redirects"
echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects
#echo "#Enable bad error message protection"
echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses
#echo "#Log martians (packets with impossible addresses)"
echo "1" > /proc/sys/net/ipv4/conf/all/log_martians
#echo "#Set out local port range"
echo "32768 61000" > /proc/sys/net/ipv4/ip_local_port_range
#echo "#Reduce DoS'ing ability by reducing timeouts"
echo "30" > /proc/sys/net/ipv4/tcp_fin_timeout
echo "2400" > /proc/sys/net/ipv4/tcp_keepalive_time
echo "0" > /proc/sys/net/ipv4/tcp_window_scaling
echo "0" > /proc/sys/net/ipv4/tcp_sack
#echo "#ECN disabled (some Cisco equipment do not work with this enabled)"
echo "0" > /proc/sys/net/ipv4/tcp_ecn
echo "sysctl options set."
}
####################################################################################
router() {
echo "Routing enabled for this server"
echo "1" > /proc/sys/net/ipv4/ip_forward
}
####################################################################################
norouter() {
echo "Routing disabled for this server"
echo "0" > /proc/sys/net/ipv4/ip_forward
}
#################################################
## Define Default DROP Policies
default_policy() {
/sbin/iptables -P INPUT DROP
/sbin/iptables -P FORWARD DROP
/sbin/iptables -P OUTPUT DROP
/sbin/iptables -t nat -P PREROUTING ACCEPT
/sbin/iptables -t nat -P POSTROUTING ACCEPT
}
####################################################################
## Define User Chains
user_chains() {
/sbin/iptables -N LOGDROP
/sbin/iptables -A LOGDROP -j LOG --log-tcp-options --log-ip-options
--log-prefix '[IPTABLES DROP] : '
/sbin/iptables -A LOGDROP -j DROP
/sbin/iptables -N LOGACCEPT
/sbin/iptables -A LOGACCEPT -j LOG --log-tcp-options --log-ip-options
--log-prefix '[IPTABLES ACCEPT] : '
/sbin/iptables -A LOGACCEPT -j ACCEPT
}
####################################################################
##Default Ruleset
default_rules() {
##Allow Loopback Interface access
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A OUTPUT -j ACCEPT
/sbin/iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A FORWARD -o $EXT_DEV -s $INT_NET -m state --state NEW
-j ACCEPT
## Allow pings from $ALLOWPING
for ALLOWPING in $ALLOWPING; do
echo "ALLOWPING $ALLOWPING"
/sbin/iptables -A INPUT --source $ALLOWPING -p icmp -m state --state
NEW -j ACCEPT
done
##Allow SSH Connections
for ALLOWSSH_INT_DEV in $ALLOWSSH_INT_DEV; do
echo "ALLOWSSH_INT_DEV(ssh port 22, tcp) $ALLOWSSH_INT_DEV -i $INT_DEV"
/sbin/iptables -A INPUT -i $INT_DEV -p tcp --source
$ALLOWSSH_INT_DEV --destination-port 22 -j ACCEPT
done
## allow dns querries from $ALLOWDNS_INT_DEV
for ALLOWDNS_INT_DEV in $ALLOWDNS_INT_DEV; do
echo "ALLOWDNS_INT_DEV(domain port 53, udp) $ALLOWDNS_INT_DEV -i
$INT_DEV"
/sbin/iptables -A INPUT -i $INT_DEV --source $ALLOWDNS_INT_DEV
--protocol udp --destination-port domain -j ACCEPT
done
#bootps 67/tcp # BOOTP server
#bootps 67/udp
#bootpc 68/tcp # BOOTP client
#bootpc 68/udp
if [ ${ALLOWDHCP_INT_DEV} ]; then
## allow dhcp querries from $ALLOWDHCP_INT_DEV
for ALLOWDHCP_INT_DEV in $ALLOWDHCP_INT_DEV; do
echo "ALLOWDHCP_INT_DEV(dhcp port 67,68, tcp,udp) $ALLOWDHCP_INT_DEV
-i $INT_DEV"
/sbin/iptables -A INPUT -i $INT_DEV -p tcp --source
$ALLOWDHCP_INT_DEV --destination-port bootps -j ACCEPT
/sbin/iptables -A INPUT -i $INT_DEV -p udp --source
$ALLOWDHCP_INT_DEV --destination-port bootps -j ACCEPT
/sbin/iptables -A INPUT -i $INT_DEV -p tcp --source
$ALLOWDHCP_INT_DEV --destination-port bootps -j ACCEPT
/sbin/iptables -A INPUT -i $INT_DEV -p udp --source
$ALLOWDHCP_INT_DEV --destination-port bootps -j ACCEPT
done
fi
}
####################################################################
## Set up IP Masquerading/FTP connection tracking
default_nat() {
/sbin/iptables -t nat -A POSTROUTING -o $EXT_DEV -j SNAT --to $EXT_IP
/sbin/modprobe ip_nat_ftp
/sbin/modprobe ip_conntrack_ftp
##redirect outbound port 80 to 198.172.205.2:80
#echo "#redirect outbound 80 to 198.172.205.2:80"
#/sbin/iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT
--to 198.172.205.2:80
## NAT/PREROUTE
if [ ${PREROUTE22} ]; then
echo "RULE_22: PreRoute inbound port 22 on $EXT_DEV to $PREROUTE22"
/sbin/iptables -t nat -A PREROUTING -i $EXT_DEV -p tcp --sport
1024:65535 -d $EXT_IP --dport 22 -j DNAT --to-destination $PREROUTE22
/sbin/iptables -A FORWARD -i $EXT_DEV -o $INT_DEV -p tcp --sport
1024:65535 -d $PREROUTE22 --dport 22 -m state --state
NEW,ESTABLISHED,RELATED -j ACCEPT
fi
}
####################################################################
default_logging() {
## Log Drop Packets
/sbin/iptables -A INPUT -j LOG --log-tcp-options --log-ip-options
--log-prefix '[IPTABLES DROP] : '
/sbin/iptables -A FORWARD -j LOG --log-tcp-options --log-ip-options
--log-prefix '[IPTABLES DROP] : '
/sbin/iptables -A OUTPUT -j LOG --log-tcp-options --log-ip-options
--log-prefix '[IPTABLES DROP] : '
/sbin/iptables -A FORWARD -j DROP
/sbin/iptables -A INPUT -j DROP
/sbin/iptables -A OUTPUT -j DROP
}
####################################################################
stop() {
/sbin/iptables --policy INPUT ACCEPT
/sbin/iptables --policy OUTPUT ACCEPT
/sbin/iptables --policy FORWARD ACCEPT
/sbin/iptables -t nat --policy PREROUTING ACCEPT
/sbin/iptables -t nat --policy OUTPUT ACCEPT
/sbin/iptables -t nat --policy POSTROUTING ACCEPT
/sbin/iptables -t mangle --policy PREROUTING ACCEPT
/sbin/iptables -t mangle --policy OUTPUT ACCEPT
/sbin/iptables --flush
/sbin/iptables -t nat --flush
/sbin/iptables -t mangle --flush
/sbin/iptables --delete-chain
/sbin/iptables -t nat --delete-chain
/sbin/iptables -t mangle --delete-chain
/sbin/iptables --zero
}
####################################################################
case "$1" in
stop)
norouter
stop
echo "iptables stopped"
;;
*)
flush
proc
router
default_policy
user_chains
default_rules
default_nat
default_logging
echo "wan side = $EXT_DEV: $EXT_IP Network: $EXT_NET"
echo "lan side = $INT_DEV: $INT_IP Network: $INT_NET"
echo "$0 Done. $DATE Base config initiated."
;;
esac
#EOF
.login
[apple:~] karl% cat .login
# Build path, checking that each exists before adding
set pathdirs = ( /bin /usr/bin /sbin /usr/sbin /usr/local/bin /usr/local/sbin /usr/X11R6/bin /usr/bin/X11 /usr/contrib/bin/X11 /sw/bin /sw/sbin ${HOME}/bin /Applications /Applications/Utilities /usr/X386/bin /usr/openwin/bin /usr/dt/bin )
set path = ()
foreach d ($pathdirs)
if ( -d $d ) set path = ( $path $d )
end
unset pathdirs d
#Alias
alias ls 'ls -aGF'
alias rm 'mv \!* ~/.Trash'
Friday, October 07, 2005
cygwin ssh server
ssh localhost
gets connection refused
ssh-host-config
privilege separation = no "nah, don't need it right now."
ssh as service = yes
CYGWIN=ntsec tty
'net start sshd' or 'cygrunsrv -S sshd'
ssh locahost
works now.
