I wrote a couple tools in perl, for CSV data processing:
dbicsv - can feed CSV data into or out from a database
csvproc – can process CSV data using a perl one-liner
Please let me know if you find these tools useful, to suggest features, or if you’re interested but don’t see how to use them. Don’t let my humility deceive you; these are very useful tools!
Some examples:
export DB_DBMS=postgrsql DB_NAME=test
dbicsv 'select * from foo;' >foo.csv
dbicsv 'insert into foo values (?, ?, ?);' <foo.csv
dbicsv 'select * from sp_proc($Col1, $Col2);' <in.csv >out.csv
csvproc '$name = uc $name' <in.csv >out.csv
csvproc -out id,name,date 'fix_date($date)' <in.csv >out.csv
csvproc -filter '$name = lc $name; $salary >= 1e6;' <in.csv >out.csv
how to use dbicsv:
Usage: dbicsv [options] sql-query [param ...]
Runs an SQL query, returns CSV data.
option description default
-dbms ? DBMS, determines DSN $DB_DBMS || InterBase
-dsn ? DBI data source name $DB_DSN || (from DBMS)
-db ? database name $DB_NAME
-user ? database user $DB_USER || $ISC_USER || "sysdba"
-pass ? database password $DB_PASS || $ISC_PASSWORD || "sysdba"
-host ? database hostname $DB_HOST
-port ? database port $DB_PORT
-in input data from stdin $DB_IN
-head ? column head = -|O|I|IO $DB_HEAD || IO
-debug show more info $DB_DEBUG
-help show this message
how to use csvproc:
Usage: csvproc [options] mapping
Process CSV data with a small perl program.
option description
-filter filter lines, mapping returns boolean
-out a,b output fields a, b (can declare new fields)
-help show this message
-debug show debug info including the generated perl code
utility function examples:
zero_pad($Var, 4);
fix_date($Date);
