Find text in multiple files

A grep alternative in Windows: findstr

An example:

findstr /i/n “DCOW” *.gms

View/create LXI files without the GAMS IDE

Shiro Takeda went one step further in making a better alternative to GAMS IDE. He added a new functionality to his gams emacs mode. From now on, you can view and create LXI files within emacs.

To invoke the LXI mode press C-c C-x (it’s strange that there’s no menu item for it, so it’s good to keep in mind this combination). The LXI file helps you to navigate through the gams list file. It contains the information where to find the most interesting/important parts (solution reports, variables, equations etc.). When started the LXI mode shows a tree structure on the left-hand side. This gives you an overview and an easy way to walk through the list file by clicking on those parts of the tree that you’re interested in.

Together with the outline mode (which is in my opinion better in investigating data and results) you have flexibile tools to understand what your model tells you.

Text editors for GAMS coding

Recently I’m working in a very restricted Windows environment, and so I needed a portable text editor that requires no installation (as I don’t have the permissions to add softwares to the system).

My favourite so far was the portable edition of Gvim (linux users know what I’m talking about). But today I found PSPad - a really handy editor with tons of features. The best in these editors is that you can add syntax highlighting rules, and then you can use them to write code in less known languages (e.g GAMS).

Update 1: I found it in an old post but I post it because I was looking for a good line wrap setting for gvim for quite some time. To set a nice word wrapping in gvim:

:set wrap linebreak textwidth=0

You can find my GAMS syntax files for these two applications below. Feel free to modify it if you like.

1. Syntax file for gvim. Instructions: simply save the code as ‘gams.vim’ and place it into the

\GVimPortable\App\vim\vim71\syntax

folder

" Vim syntax file
" Language:        gams

" For version 5.x: Clear all syntax items
" For version 6.x: Quit when a syntax file was already loaded
if version < 600
  syntax clear
elseif exists("b:current_syntax")
  finish
endif

" A bunch of gams keywords
syn keyword gamsStatement       Sets Set SET SETS set sets
syn keyword gamsStatement       Table table TABLE
syn keyword gamsStatement       Parameter Parameters PARAMETER PARAMETERS
syn keyword gamsStatement       parameter parameters
syn keyword gamsStatement       Model model MODEL model
syn keyword gamsStatement       Solve solve SOLVE
syn keyword gamsStatement       Equations Equation equation EQUATIONS
syn keyword gamsStatement       equations
syn keyword gamsStatement       Variable variable Variables variables
syn keyword gamsStatement       VARIABLE VARIABLES
syn keyword gamsStatement       $option $OPTION $Option
syn keyword gamsStatement       display DISPLAY Display
syn keyword gamsStatement       option Option
syn keyword gamsStatement       Alias alias
syn keyword gamsStatement       Scalar scalar SCALAR
syn keyword gamsStatement       xxpto
" plusz utasítások
syn keyword gamsStatement       put file putclose abort
syn keyword gamsConditional     if else $ If Elseif ElseIf
syn keyword gamsRepeat          loop while repeat until
syn keyword gamsRepeat          for to Downto By For To downto by
syn keyword gamsStatement       $call $if not
"  syn keyword gamsStatement       $eval $include

syntax keyword gamsFunction     ABS abs Abs ASC asc Asc
syntax keyword gamsFunction      abs acos acosh alias asin asinh atan atan2
syntax keyword gamsFunction      atanh ceil ctime cos cosh exp floor log log10
syntax keyword gamsFunction      max min precision round sin sinh sqrt tan tanh
syntax keyword gamsFunction      time trunc div

syntax keyword gamsTodo contained       TODO

syn match gamsSpecial           "^\$\<[^ ]\+\>"

"integer number, or floating point number without a dot.
syn match  gamsNumber           "\<\d\+\>"
"floating point number, with dot
syn match  gamsNumber           "\<\d\+\.\d*\>"
"floating point number, starting with a dot
syn match  gamsNumber           "\.\d\+\>"

"integer number, or floating point number without a dot.
syn match  gamsNumber           "\<\d\+\>"
"floating point number, with dot
syn match  gamsNumber           "\<\d\+\.\d*\>"
"floating point number, starting with a dot
syn match  gamsNumber           "\.\d\+\>"

" String and Character contstants
syn region  gamsString            start=+"+  skip=+\\\\\|\\"+  end=+"+

syn match   gamsComment         "#.*$"
syntax match gamsComment       "^\*.*"
"syn match gamsComment "^/star.*$"
"syn region gamsComment start="^\*" end="^\*"
syntax region  gamsComment         start="^\$ontext" end="^\$offtext"
" syn match   gamsMathsOperator   "-\|=\|[:<>+\*^/\\]\|AND\|OR"
syn match   gamsMathsOperator   "-\|=\|[:<>+^/\\]\|AND\|OR"

syn region gamsInclude start="\$ondelim" end="\$offdelim"
"syn region gamsInclude start="%" end="%"
syn region gamsInclude start="\"%" end="%\""

" Define the default highlighting.
" For version 5.7 and earlier: only when not done already
" For version 5.8 and later: only when an item doesn't have
" highlighting yet
if version >= 508 || !exists("did_gams_syntax_inits")
  if version < 508
    let did_gams_syntax_inits = 1
    command -nargs=+ HiLink hi link <args>
  else
    command -nargs=+ HiLink hi def link <args>
  endif

  HiLink gamsLabel              Label
  HiLink gamsConditional        Conditional
  HiLink gamsRepeat             Repeat
  HiLink gamsLineNumber Comment
  HiLink gamsNumber             Number
  HiLink gamsError              Error
  HiLink gamsStatement  Statement
  HiLink gamsString             String
  HiLink gamsComment            Comment
  HiLink gamsSpecial            Special
  HiLink gamsTodo               Todo
  HiLink gamsFunction           Identifier
  HiLink gamsTypeSpecifier Type
  HiLink gamsFilenumber gamsTypeSpecifier
  HiLink gamsInclude           Special
  hi gamsMathsOperator term=bold cterm=bold gui=bold

  delcommand HiLink
endif

let b:current_syntax = "gams"

" vim: ts=8

2. Syntax file for PSPad. Save the code as ‘GAMS.ini’ in the ‘\pspad\syntax’ folder, then you can configure the colors in the ‘Settings\Highlighters settings’ menu.

;PSPad user HighLighter definition file
[Settings]
Name=GAMS
HTMLGroup=0
Label=1
FileType=.gms,.lst
CommentString=*
PocoComment=1
2StarsComment=1
IndentChar={
UnIndentChar=}
TabWidth=4
SingleQuote=1
DoubleQuote=1
KeyWordChars=-_$
CodeExplorer=ftUnknown
[KeyWords]
ABS=
acos=
acosh=
Alias=
asin=
asinh=
atan=
atan2=
atanh=
ceil=
cos=
cosh=
ctime=
display=
div=
Equations=
equations=
exp=
floor=
log=
log10=
max=
min=
Model=
option=
Parameter=
Parameters=
precision=
round=
Scalar=
Set=
sets=
sin=
sinh=
Solve=
sqrt=
Table=
tables=
tan=
tanh=
time=
trunc=
uniform=
Variable=
variables=
[ReservedWords]
[KeyWords2]
abort=
By=
Downto=
else=
Elseif=
file=
for=
if=
loop=
put=
putclose=
repeat=
to=
until=
while=
xxpto=
[KeyWords3]
$call=
$eval=
$if not=
$include=
$option=

Debugging GAMS code

It’s good to feed Erwin Kalvelagen’s blog into your rss reader, because you can always find some useful GAMS tricks there. Or if you have some free time, read the longer posts dealing with interesting topics in optimization.

Erwin has recently published a post about debugging GAMS codes.

An advice of him is to use the $stop statement to stop the execution in a certain point and then save the current state of the model into a gdx file. He also talks about the difficulties of writing loops in GAMS.

Here you can find some additional info about gdx files and creating loops.

Update: Another gams insider’s blog is here.

My new project is released!

This post is really an announcement. In a recent post I talked about a project I was working on, and later I was asked in a comment whether the code would be published. This question was the final incentive to register for a sourceforge site and let the project go.

The project aims to develop a small utility that converts gdx data files into open office spreadsheets. It’s meant to be an alternative of the existing gdx2xls utility for Linux users. Now, we Linux users are able to convert our data files into human readable spreadsheets.

So, check out my new project on sourceforge.net if you’re interested!

Comments, suggestions, remarks are welcome, as always.

DVD+RW doesn’t need to be erased

I accidentally ran into this issue when I wanted to reuse a DVD+RW disk: DVD+RW disks do not support blanking (erasing). You only have to overwrite them without erasing.

When I inserted a DVD+RW in the drive and wanted to erase it with brasero, the application sent me a pop-up window with an error message and ejected the disk. To find out what happened I opened a terminal and typed:

cdrecord dev=/dev/dvd blank=all

then I got the error message:

cdrecord: OPC failed.
Error: this media does not support blanking, ignoring.
This drive or media does not support the ‘BLANK media’ command
cdrecord: Cannot blank disk, aborting.

After googling it a bit I found this post that calmed me down. DVD+RW’s are only need to be overwritten. So I simply started brasero again and burned a new project onto the disk.

Convert avi to dvd with command line tools

Assume you have a divx movie (e.g. sneakers.avi) and two subtitle files (sneakersHUN.srt, sneakersENG.srt) that you want to convert into dvd format. In this blog entry I’ll only use command line tools. This is because there’s no need to write another “How to convert avi to dvd with Devede” article, and because most GUI tools available in Ubuntu are simple graphical frontends of ffmpeg, mencoder and dvdauthor (the command line tools described bellow).

Let’s get down to business:

1. Convert your .avi file with ffmpeg or mencoder to a suitable .mpg format

Using ffmpeg is less complicated because it needs much less command line options. As an example (converting an avi file into a pal dvd format):

ffmpeg -i film.avi -aspect 4:3 -target pal-dvd dvd.mpg

If using mencoder, a lot of command line options have to be added. To make my life easier, I use a bash script in that I can simply change the file name, aspect ratio and what else is needed. Here’s the script:

#! /bin/bash
# bash script for creating a pal dvd
# movie is sneakers!

mencoder -oac lavc -ovc lavc -of mpeg -mpegopts format=dvd:tsaf \
-vf scale=720:576,harddup -srate 48000 -af lavcresample=48000 \
-lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=5000:\
keyint=15:vstrict=0:acodec=ac3:abitrate=192:aspect=16/9 -ofps 25 \
-o sneakers.mpg sneakers.avi

Notes: the DVD format is heavily constrained, i.e. bitrates and screen sizes are pre-defined. Take a look at this table.

2. Add the subtitles with spumux

• Create two xml files with the contents below, and save them as “spumuxHUN.xml” and “spumuxENG.xml” (one for each language) .

  <subpictures>
      <stream>
         <textsub filename="sneakersHUN.srt" characterset="ISO8859-2"
                      fontsize="28.0" font="devedesans.ttf" horizontal-alignment="left"
                      vertical-alignment="bottom" left-margin="60" right-margin="60"
                      top-margin="20" bottom-margin="30" subtitle-fps="25"
                      movie-fps="25" movie-width="720" movie-height="574"
                      force="yes"
          />
       </stream>
  </subpictures>
  

  Notes: the xml files contain the necessary information for spumux. Option names speak for themselves. One remark to “font=”: you can use whatever fonts you like, only copy the .ttf file into the ~/.spumux directory (devedesans.ttf is the only font in Ubuntu that’s available by default).

• Create an mpg file with the subtitles embedded:
  

  spumux -s0 spumuxHUN.xml sneakersHUN.mpg
  rm sneakers.mpg
  spumux -s1 spumuxENG.xml sneakersHUNENG.mpg
  rm sneakersHUN.mpg

3. Create the dvd file system with dvdauthor in two steps:

• Create a simple text file with the following content and save it as auth.xml

  <dvdauthor>
      <vmgm />
      <titleset>
          <titles>
              <subpicture lang="hu"/>
              <subpicture lang="en" />
              <pgc>
                  <vob file="sneakersHUNENG.mpg" />
              </pgc>
          </titles>
      </titleset>
  </dvdauthor>
  

• type the following into a terminal:
  

  dvdauthor -o mydvd -x auth.xml

  Note: this will create the dvd filesystem in the mydvd folder.

  4. Burn the dvd! A simple command line tool to do this:

  growisofs -Z /dev/dvd -dvd-video mydvd/

  Be aware that the -Z option will erase the disk if it’s not empty!

Extracting .daa images on Ubuntu

The most simple solution I found so far to extract an image file in .daa format  is:

1. Download PowerISO from here.
2. Extract it and add the folder to the PATH.
3. Open terminal and type:

   poweriso extract disk.daa / -od /destination/folder

To see all the poweriso options and some examples of usage, type:

poweriso -?

To look inside the .daa file:

poweriso list disk.daa / -r

Evolution keeps asking for password of the default keyring

After updating to a newer Ubuntu version and changing root password, Evolution mail kept asking for a password to gain access to the default keyring. It means I had to enter my old root password every time Evolution started. It’s because this password was bounded to the old keyring, located at:

~/.gnome2/keyrings/login.keyring

To fix this issue simply delete the login.keyring and restart Evolution. After relaunching, Evolution asks a new password and generate a new default keyring (~/.gnome2/keyrings/default.keyring).

Samba and MySQL administration

SAMBA

To set the samba password:

sudo smbpasswd -a [username]

The option “-a” can be left out if the samba user already exists. Note that [username] is the Linux/Unix username. That can be different from the samba user name. Aliases can be inserted into a separate text file (e.g.  /etc/samba/smbusers):

username = “samba_username”

If you do so than edit smb.conf to find your text file – insert into smb.conf the following:

username map = /etc/samba/smbusers

MySQL

to change your password without root privileges: (1) first log in than (2) enter the following:

set password = password(‘new_password’);

Don’t forget to update the [client] section of your .my.cnf file. For security reasons, always change the permissions of .my.cnf:

sudo chmod 400 ~/.my.cnf