Problem:

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;
createdb: database creation failed: ERROR:  encoding UTF8 does not match locale en_US
DETAIL:  The chosen LC_CTYPE setting requires encoding LATIN1.

1
2
3

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;
createdb: database creation failed: ERROR:  encoding UTF8 does not match locale en_US
DETAIL:  The chosen LC_CTYPE setting requires encoding LATIN1.

Check what locales you have

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
POSIX

1
2
3
4
5
6

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
POSIX

If locales are not installed:

# apt-get install debconf

1

# apt-get install debconf

To add en_US.UTF8, run:

# dpkg-reconfigure locales

1

# dpkg-reconfigure locales

And select the locales you want (en_US.UTF8)

Now:

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
en_US.utf8
POSIX

1
2
3
4
5
6
7

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
en_US.utf8
POSIX

You need to restart postgres:

# /usr/lib/postgresql/8.4/bin/pg_ctl restart -D /var/lib/postgresql/8.4/main

1

# /usr/lib/postgresql/8.4/bin/pg_ctl restart -D /var/lib/postgresql/8.4/main

Now rerun:

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;

1

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;

Spent a bit too long fixing a sudo problem right after an apt-get upgrade on a Debian machine.

The original problem:

# sudo ls
sudo: unable to initialize PAM: No such file or directory

1
2

# sudo ls
sudo: unable to initialize PAM: No such file or directory

Sudo errors are logged with syslog by default. Some message should be in one of the /var/log/xx.log files. If you don’t know which, run

# cd /var/log
# grep pam *.log
auth.log:Nov 21 22:35:14 mach sudo: PAM _pam_load_conf_file: unable to open /etc/pam.d/common-session-noninteractive
auth.log:Nov 21 22:35:14 mach sudo: PAM _pam_init_handlers: error reading /etc/pam.d/sudo
auth.log:Nov 21 22:35:14 mach sudo: PAM _pam_init_handlers: [Critical error - immediate abort]
auth.log:Nov 21 22:35:14 mach sudo: PAM pam_start: failed to initialize handlers
auth.log:Nov 21 22:35:14 mach sudo: PAM pam_end: NULL pam handle passed

1
2
3
4
5
6
7

# cd /var/log
# grep pam *.log
auth.log:Nov 21 22:35:14 mach sudo: PAM _pam_load_conf_file: unable to open /etc/pam.d/common-session-noninteractive
auth.log:Nov 21 22:35:14 mach sudo: PAM _pam_init_handlers: error reading /etc/pam.d/sudo
auth.log:Nov 21 22:35:14 mach sudo: PAM _pam_init_handlers: [Critical error - immediate abort]
auth.log:Nov 21 22:35:14 mach sudo: PAM pam_start: failed to initialize handlers
auth.log:Nov 21 22:35:14 mach sudo: PAM pam_end: NULL pam handle passed

One configuration file was missing. This PAM debian page told me what the file was supposed to contain. Somehow that file was not created in the upgrade.

# cat /etc/pam.d/common-session-noninteractive
session		[default=1]						pam_permit.so
session		requisite						pam_deny.so
session		required						pam_permit.so
session		required						pam_unix.so

1
2
3
4
5

# cat /etc/pam.d/common-session-noninteractive
session		[default=1]						pam_permit.so
session		requisite						pam_deny.so
session		required						pam_permit.so
session		required						pam_unix.so

Fixed.

Last Friday I came back from a 10 day road trip driving from San Francisco to Seattle, and coming back with the Amtrak Coast Starlight train – as I mentioned earlier.

I used Foursquare a bit more than usual (about 50 checkings over 10 days) and took a lot of pictures of rocks, landscapes, forests, the ocean, etc.
I started making a simple mashups to visualize the overall itinerary I ended up doing.

View full screen (Pink bubbles have photos. Blue bubbles are the checkins on the way to Seattle. Yellow bubbles are the checkins on the way back to San Francisco.)

I have photos on Flickr.

Some highlights:

• 1990 miles / 3200 km driven from SF to Seattle in 8 days.
• Very nice weather except the day I drived through Crater Lake, which I could not see at all because of the fog.
• Not many tourists or anyone on the roads end of October.
• I loved all the coastal areas, and especially the Olympic National Park.
• I ended up staying in campgrounds most of the nights, in the middle of a forest or close to the ocean, with barely anyone around. That was awesome.
• It’s not easy to find the cheap / non-RV campgrounds where you can put a tent or just sleep in the car.
• All the roads were surprisingly well maintained (I’m too used to the bad roads of south SF).
• I really liked the cities of Bend and Portland in Oregon. Seattle, WA… not as much but that was at the end of my trip. Redding, CA was weird.
• I think I expected a bit more from the train trip (23 hours from Seattle to San Francisco). That was really comfortable but as I was into a good book, and we were going through areas I had driven across, I did not pay much attention to the landscape.

I put the code for the mashup on github.

• Install the packages:

% pip install FormAlchemy fa.query GeoFormAlchemy

1

% pip install FormAlchemy fa.query GeoFormAlchemy

• You have now more templates available:

% paster create --list-templates
  basic_package:   A basic setuptools-enabled package
  geo_fa:          Pylons application template with GeoFormAlchemy support
  mapfish:         MapFish application template
  mapfish_client:  MapFish client plugin template
  paste_deploy:    A web application deployed through paste.deploy
  pylons:          Pylons application template
  pylons_fa:       Pylons application template with formalchemy support
  pylons_minimal:  Pylons minimal application template

1
2
3
4
5
6
7
8
9

% paster create --list-templates
  basic_package:   A basic setuptools-enabled package
  geo_fa:          Pylons application template with GeoFormAlchemy support
  mapfish:         MapFish application template
  mapfish_client:  MapFish client plugin template
  paste_deploy:    A web application deployed through paste.deploy
  pylons:          Pylons application template
  pylons_fa:       Pylons application template with formalchemy support
  pylons_minimal:  Pylons minimal application template

• You can run the command to add some of the GeoFormAlchemy or the formalchemy support to your project:

% paster create -t geo_fa

1

% paster create -t geo_fa

Keep the default values except for the first one

Enter admin_controller (Add formalchemy's admin controller) [False]: True 

1

Enter admin_controller (Add formalchemy's admin controller) [False]: True

And answer no unless  you want to overrite a file:

Overwrite ./MyApp/myapp/config/environment.py [y/n/d/B/?] n
[etc ...]

1
2

Overwrite ./MyApp/myapp/config/environment.py [y/n/d/B/?] n
[etc ...]

• If you do not run the paster create command, you’ll have to add those files directly:
• Add the file

  myapp/controllers/admin.py

• Add the file

  myapp/forms/__init__.py

• Add the template files in (on githhub-FormAlchemy and github-GeoFormAlchemy)

  myapp/templates/forms/ fieldset.mako,
  fieldset_readonly.mako, grid.mako, grid_readonly.mako, map.mako, map_js.mako,
  restfieldset.mako

• Modify

  myapp/config/routing.py

• Add the admin controller

  myapp/controllers/admin.py

### myapp/controllers/admin.py
import logging
from formalchemy.ext.pylons.controller import ModelsController
from webhelpers.paginate import Page
from myapp.lib.base import BaseController, render
from myapp import model
from myapp import forms
from myapp.model import meta

log = logging.getLogger(__name__)

class AdminControllerBase(BaseController):
    model = model # where your SQLAlchemy mappers are
    forms = forms # module containing FormAlchemy fieldsets definitions
    def Session(self): # Session factory
        return meta.Session

    ## customize the query for a model listing
    # def get_page(self):
    #     if self.model_name == 'Foo':
    #         return Page(meta.Session.query(model.Foo).order_by(model.Foo.bar)
    #     return super(AdminControllerBase, self).get_page()

AdminController = ModelsController(AdminControllerBase,
                                   prefix_name='admin',
                                   member_name='model',
                                   collection_name='models',
                                  )

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

### myapp/controllers/admin.py
import logging
from formalchemy.ext.pylons.controller import ModelsController
from webhelpers.paginate import Page
from myapp.lib.base import BaseController, render
from myapp import model
from myapp import forms
from myapp.model import meta

log = logging.getLogger(__name__)

class AdminControllerBase(BaseController):
    model = model # where your SQLAlchemy mappers are
    forms = forms # module containing FormAlchemy fieldsets definitions
    def Session(self): # Session factory
        return meta.Session

    ## customize the query for a model listing
    # def get_page(self):
    #     if self.model_name == 'Foo':
    #         return Page(meta.Session.query(model.Foo).order_by(model.Foo.bar)
    #     return super(AdminControllerBase, self).get_page()

AdminController = ModelsController(AdminControllerBase,
                                   prefix_name='admin',
                                   member_name='model',
                                   collection_name='models',
                                  )

• Modify

  myapp/config/routing.py

### In myapp/config/routing.py
    # Map the /admin url to FA's AdminController
    # Map static files
    map.connect('fa_static', '/admin/_static/{path_info:.*}', controller='admin', action='static')
    # Index page
    map.connect('admin', '/admin', controller='admin', action='models')
    map.connect('formatted_admin', '/admin.json', controller='admin', action='models', format='json')
    # Models
    map.resource('model', 'models', path_prefix='/admin/{model_name}', controller='admin')

1
2
3
4
5
6
7
8
9

### In myapp/config/routing.py
    # Map the /admin url to FA's AdminController
    # Map static files
    map.connect('fa_static', '/admin/_static/{path_info:.*}', controller='admin', action='static')
    # Index page
    map.connect('admin', '/admin', controller='admin', action='models')
    map.connect('formatted_admin', '/admin.json', controller='admin', action='models', format='json')
    # Models
    map.resource('model', 'models', path_prefix='/admin/{model_name}', controller='admin')

• myapp/forms/__init__.py

### myapp/forms/__init__.py
from pylons import config
from myapp.model import my_app_points
from myapp.lib.base import render
from formalchemy import config as fa_config
from formalchemy import templates
from formalchemy import validators
from formalchemy import fields
from formalchemy import forms
from formalchemy import tables
from formalchemy.ext.fsblob import FileFieldRenderer
from formalchemy.ext.fsblob import ImageFieldRenderer

from geoformalchemy.base import GeometryFieldRenderer
from geoalchemy import geometry
forms.FieldSet.default_renderers[geometry.Geometry] = GeometryFieldRenderer

fa_config.encoding = 'utf-8'
class TemplateEngine(templates.TemplateEngine):
    def render(self, name, **kwargs):
        return render('/forms/%s.mako' % name, extra_vars=kwargs)
fa_config.engine = TemplateEngine()
class FieldSet(forms.FieldSet):
    pass
class Grid(tables.Grid):
    pass
## Initialize fieldsets
#Foo = FieldSet(model.Foo)
#Reflected = FieldSet(Reflected)
## Initialize grids
#FooGrid = Grid(model.Foo)
#ReflectedGrid = Grid(Reflected)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

### myapp/forms/__init__.py
from pylons import config
from myapp.model import my_app_points
from myapp.lib.base import render
from formalchemy import config as fa_config
from formalchemy import templates
from formalchemy import validators
from formalchemy import fields
from formalchemy import forms
from formalchemy import tables
from formalchemy.ext.fsblob import FileFieldRenderer
from formalchemy.ext.fsblob import ImageFieldRenderer

from geoformalchemy.base import GeometryFieldRenderer
from geoalchemy import geometry
forms.FieldSet.default_renderers[geometry.Geometry] = GeometryFieldRenderer

fa_config.encoding = 'utf-8'
class TemplateEngine(templates.TemplateEngine):
    def render(self, name, **kwargs):
        return render('/forms/%s.mako' % name, extra_vars=kwargs)
fa_config.engine = TemplateEngine()
class FieldSet(forms.FieldSet):
    pass
class Grid(tables.Grid):
    pass
## Initialize fieldsets
#Foo = FieldSet(model.Foo)
#Reflected = FieldSet(Reflected)
## Initialize grids
#FooGrid = Grid(model.Foo)
#ReflectedGrid = Grid(Reflected)

myapp/model/my_app_points.py

###myapp/model/my_app_points.py
from sqlalchemy import Column, types
from geoalchemy import GeometryColumn, Point
from mapfish.sqlalchemygeom import GeometryTableMixIn
from mapfishapp.model.meta import Session, Base

class MyAppPoint(Base, GeometryTableMixIn):
    autoload_with = Session.bind
    __tablename__ = 'my_app_point'
    __table_args__ = {
        "autoload_with": Session.bind,
        "autoload": True,
    }
    way = GeometryColumn(Point(srid=900913))

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

###myapp/model/my_app_points.py 
from sqlalchemy import Column, types
from geoalchemy import GeometryColumn, Point
from mapfish.sqlalchemygeom import GeometryTableMixIn
from mapfishapp.model.meta import Session, Base

class MyAppPoint(Base, GeometryTableMixIn):
    autoload_with = Session.bind
    __tablename__ = 'my_app_point'
    __table_args__ = {
        "autoload_with": Session.bind,
        "autoload": True,
    }
    way = GeometryColumn(Point(srid=900913))

I’d get an error by importing that class in

myapp/model/__init__.py

from myapp.models.my_app_points import MyAppPoint

1

from myapp.models.my_app_points import MyAppPoint

would lead to that error:

sqlalchemy.exc.UnboundExecutionError: No engine is bound to this Table's MetaData.
Pass an engine to the Table via autoload_with=<someengine>, or associate the MetaData
with an engine via metadata.bind=<someengine>

1
2
3

sqlalchemy.exc.UnboundExecutionError: No engine is bound to this Table's MetaData.
Pass an engine to the Table via autoload_with=<someengine>, or associate the MetaData
with an engine via metadata.bind=<someengine>

Modifying myapp/model/__init__.py by adding Base.metadata.bind = engine at the end of init_model(engine) did not work.

The solution I found after checking the formalchemy code was to modify the myapp/controllers/admin.py and pass a list of the model classes to list in the web admin view.

###Modify in myapp/controllers/admin.py
from myapp.model.my_app_points import MyAppPoint

list_models = [MyAppPoint,]
class AdminControllerBase(BaseController):
    model = list_models # where your SQLAlchemy mappers are
    forms = forms # module containing FormAlchemy fieldsets definitions

1
2
3
4
5
6
7

###Modify in myapp/controllers/admin.py
from myapp.model.my_app_points import MyAppPoint

list_models = [MyAppPoint,]
class AdminControllerBase(BaseController):
    model = list_models # where your SQLAlchemy mappers are
    forms = forms # module containing FormAlchemy fieldsets definitions

And finally you can add something like that at the end of your

myapp/forms/__init__.py

MyAppPoint = FieldSet(my_app_points.MyAppPoint)
MyAppPoint.configure(options=[MyAppPoint.way.label('Geometry').required()])
MyAppPoint.way.set(options=[
        ('map_srid', 900913),
        ('base_layer', 'new OpenLayers.Layer.OSM("OSM")'),
        ('openlayers_lib', '/lib/openlayers/lib/OpenLayers.js'),
        ('zoom', 14)
    ])

1
2
3
4
5
6
7
8

MyAppPoint = FieldSet(my_app_points.MyAppPoint)
MyAppPoint.configure(options=[MyAppPoint.way.label('Geometry').required()])
MyAppPoint.way.set(options=[
        ('map_srid', 900913),
        ('base_layer', 'new OpenLayers.Layer.OSM("OSM")'),
        ('openlayers_lib', '/lib/openlayers/lib/OpenLayers.js'),
        ('zoom', 14)
    ])

Sources: mapfish-geoformalchemy,

It seems mapnik is getting easier and easier to setup. I wanted to reinstall mapnik2 on a mac bloated with different versions of python, of macports… I just left my install of postgres/postgis/osm, and cleaned the system before installing mapnik.

You can still expect to run into a frew trouble and may have to update the formulas.

Macports (if installed)

Careful with the rm…

Get rid of macports

%% sudo port -fp uninstall --follow-dependents installed

–edit– Check that you do not have software installed in /opt/local installed through non macport means:

%% sudo rm -rf \
/opt/local \
/Applications/DarwinPorts \
/Applications/MacPorts \
/Library/LaunchDaemons/org.macports.* \ /Library/Receipts/DarwinPorts*.pkg \
/Library/Receipts/MacPorts*.pkg \ /Library/StartupItems/DarwinPortsStartup \
/Library/Tcl/darwinports1.0 \
/Library/Tcl/macports1.0 \
~/.macports

Clean up the system (Python)

%% sudo rm -rf /System/Library/Frameworks/Python.framework \
/usr/bin/python* \
/Library/Frameworks/Python.framework \
%% sudo rm -rf /usr/local/lib/python \
/usr/bin/easy_install* \
/usr/bin/pydoc*


Clean previous installation of GEOS/Mapnik/etc

%% sudo rm -rf /Library/Frameworks/GDAL.framework \
/Library/Frameworks/FreeType.framework \
/Library/Frameworks/GEOS.framework \
/Library/Frameworks/Mapnik.framework \
/Library/Frameworks/PROJ.framework \
/Library/Frameworks/UnixImageIO.framework

Install Homebrew

Install Homebrew

Run

%% brew doctor
%% brew update

And fix the issues if any, like adding /usr/local/sbin and /usr/local/bin to your PATH.

Install Formula

%% brew install python

Modify your  PATH (.bash_profile) to add /usr/local/share/python

Install GDAL

%% easy_install numpy
%% brew install gdal

You may have to fix some things. I had to install GEOS with

%% brew install --use-gcc geos

Install icu4c

%% brew install icu4c

Install boost

%% brew install -v boost

(It’s ok if boost is compiled without ICU support)

Install Mapnik: (You might have to update your formula with this - brew edit mapnik)

%% brew install -v mapnik

-v gives you more information about the install. If for instance pg_config is not found, add it to PATH.

To test your installation of mapnik, you can run the mapnik/demo/python/run_demo.py script (from the mapnik-2.x.x.tar.gz install file, which is by default in ~/Library/Caches/Homebrew/)

What a nice coincidence: One year that this blog had not seen any fresh and spirited content. No longer! I have reclaimed a bit of my freedom by leaving my job at Nokia a week ago, and I’m now in a position that I can do whatever I like. What should that be, you ask me? A bit of traveling (the US is big, Europe is far), a bit of hacking here and there, and massive amount of meditation in a remote desert (I’m kidding).
It’s been a long time that I have not had a longer break than 3 weeks of vacation (I know that it’s forever for many here), and I’m really looking forward to getting bored. But then again, if you live in the Bay Area and you get bored, you’re not doing it right.

I’m actually leaving on a 10 day road trip tomorrow morning. I’ll be driving up along the coast of Northern California to Eureka, east towards Lassen Park and Mt Shasta, north again in Oregon territory, zigzagging among the forest park lands to Portland, Bend, then to Seattle. A good friend helped me make the plan, arguing among other things for the mandatory sight of large amount of solidified lava.
The route looks like this (Google maps) at the moment.

I then drop the rental in Seattle, and take the Coast Starlight train (wifi included, thank you Amtrak) all the way down to the Bay Area in one stretch of 22 hours.

Truth and lies and lives are leaking all over the Internet these days. Be it the Wikileaks reports, or the special report of a Duke student, the daily Facebook tales of misfortune… today’s confessions are done on the web, and very smart who can tell who the biggest sinner is. I would not be surprised Moms are telling their kids “be good, or the Internet will know about it, and you’ll be punished!”.

The days of an almighty power scaring the shit out of people are gone. Is anybody scared of God anymore? I am guessing those who have been on the Internet’s receiving end would take God punishment over the very special and very viral attention ‘It’ is giving them. There is no purgatory in the Internet, what gets there, stays there, and most likely for a long time. Enough time to embarrass you in all the decades of your life, and those of your kids and grand-kids. Internet’s memory is infinite and its ways are not that inscrutable.

It sucks, and it’s probably unfair for a lot of those unlucky bastards. There used to be a time when you could be stupid, foolish, and even dead mad, and get away with it. If human justice did not take care of you, certainly God’s would – once you’re dead. Today, not even the high ranking officials of the top military power on this planet are safe from Internet’s scrutiny. I suppose that’s what they call sousveillance.

Now I’m wondering: when will people switch to the Internet for their source of morality? If you start acting moral because you think the Internet is watching you, then that might be it!

The idea of the singularity starts with numbers. Numbers that get bigger and bigger over time, exponentially. As Ray Kurzweil showed it in his presentation, big numbers getting bigger are all over the technological world:

• Moore’s law
• Total bits shipped
• Magnetic data storage bits per dollar
• Internet data traffic
• Internet backbone bandwidth
• US nanotechnology-related patents

But also in biology, for instance the cost of sequencing DNA that is dropping exponentially.

Conjecturing what happens next when those numbers get really big is what this summit and the groups involved in the singularity are about. Something is bound to happen. Something special, something unforeseen, something beyond any of our wildest prediction. Will it be good (techno-optimist), bad (techno-pessimist), the same (techno-neutral) or good or bad depending on how we get there (techno-volatile)? What is going to happen, to people, to human intelligence, to artificial intelligence, to the biological world? When will the singularity point be past?

So, the singularity is where we’re going, but the work still needs to be done. And that’s what computer scientists, biologists, neuroscientists, writers, futurists, psychologists, historians and many other experts come together to discuss.

One main theme is understanding the human body and the brain better. Not just to cure diseases, repair the body, make us immortal, and more intelligent, but also to find principles that guide scientists to make smarter, more powerful technology, such as nano machines that writes and reads DNA, or computer systems that are generally intelligent.

With that many different angles and approaches to the singularity, there is no unified view of it and it is hard to even get a more precise idea about what it could be or mean. You get bits and pieces here and there, a glimpse over here. And as they say, one thing that is certain about the future is that you cannot predict it.

Besides the big ideas, was the conference good? Yes, it was nice to attend, listen to some of the guys whose blogs and articles I read from time to time. The distribution was as usual, 4-5 talks I really enjoyed and were thought-provoking to me, a few I could have slept through and the rest in the middle. I enjoyed it overall but I’m not sure experts would learn that much per se from their own field from the talks, unless you come to network.

The videos should get on the website sometime.
Read the day one and day two report articles from the h+ magazine.

Si vous venez visiter la Bay Area et San Francisco et que vous voulez gouter à l’esprit high-tech et entrepreneur de la Valley, c’est facile, il n’y a qu’à chercher un peu. Presque tous les jours, vous trouverez des meetings et meetups organisés par les nombreux groupes et associations de la région.
Quelques bonnes adresses pour commencer:

• Startup Digest: inscrivez-vous à la mailing list ou lisez leurs archives (cherchez SF Bay Area HackersDigest).
• Eventbrite: recherchez tech dans la Bay Area
• Women2.0 propose ses propres événements et tient  à jour son propre listing

La plupart des événements organisés sont soit gratuits soit peu cher ($10-$20).

This article is relatively old (from 2007) but nonetheless interesting. It basically says that a condition for higher productivity is access to intangible wealth. Whereas natural and built capital describe natural resources, pasture land, and machinery, equipment, infrastructure; intangible wealth include such things as trust in society, property rights, education, etc.  And the amount of intangible wealth is decisive for how productive and creative you will be.

Sounds intuitive enough.

Concretely, what does it mean for you? Well, if you were pondering whether you really need a new computer, a bigger screen, a lighter bike, the newest camera, you can help convince yourself that you’ll be definitely more productive, says the World Bank. Self-deception is a powerful mind trick.