Accent français

It took a while, but I finally feel I can get auto layout to do the basic things that I need it to do.
I’ve put together a test app with a few different layouts – each in a separate view controller.

My approach to work with auto layout has been:

• Define the layout and behavior of the objects to put in the view
• Add one by one the elements to the view
• Set the constraints
• Run the app to test the view
• If there is a conflict, fix the conflict
• Have a breakpoint in viewDidAppear, and check for ambiguous constraints with
  po [[UIWindow keyWindow] _autolayoutTrace]
• If there is no ambiguous layout, check the behavior with orientation and device changes (iPhone 3.5/4, iPad)
• Keep adding the views and checking for conflicts and ambiguous constraints

Some tips to debug layouts

• po [[UIWindow keyWindow] _autolayoutTrace] shows the full hierarchy of views in the app. It also indicates when a view has an ambiguous constraint
  (lldb) po [[UIWindow keyWindow] _autolayoutTrace]
$0 = 0x08d46830
*<UIWindow:0x8e476e0>
| *<UILayoutContainerView:0x8e4c820>
| | *<UITransitionView:0x8e4da90>
| | | *<UIViewControllerWrapperView:0x8e5ac70>
| | | | *<UIView:0x8e57ef0>
| | | | | *<UIScrollView:0x8e574b0>
| | | | | | *<UIView:0x8e57b70>
| | | | | | | *<View1:0x8e59fb0> - AMBIGUOUS LAYOUT
| | | | | | | *<View1:0x8e5a6d0> - AMBIGUOUS LAYOUT
• po [0x7549380 constraintsAffectingLayoutForAxis:0] shows the constraints that are used by the auto layout manager to compute the layout for that view (0x… is the address of a view that you get when printing out the trace). The parameter is 0 for horizontal axis and 1 for vertical.
  You get a list such as <NSLayoutConstraint:0x75498d0 H:|-(8)-[View1:0x7549380] (Names: '|':UIView:0x7546df0 )> . The list of view names is unfortunately not very useful for debugging iOS constraints – it’s supposed to display a string identifier that you can set for each view, but so far it works only for Mac development. For this reason, when getting stuck debugging the layout for a complex UI, it might make sense to (temporarily) subclass the views so that you at get a little bit more information when printing this debug information.
• Conflicting constraints trigger an exception when they are added to the view. Conflicting constraints are solved by the auto layout manager by removing one or more constraints until the manager can compute a layout: Will attempt to recover by breaking constraint. Forgetting to set translatesAutoresizingMaskIntoConstraints to NO for views created programmatically and added with constraints to a view often cause this.
• Conflicting constraints mean that there is no solution to compute the UI layout. For instance, the width of a view is required to be 100pts and 200pts.
• Ambiguous constraints mean that there might be several solutions to a layout. For instance, a view could be either 100pts or 200pts. The auto layout manager picks a solution, but might not pick the same one every time.
• Other debugger: po [0x123456 hasAmbiguousLayout] which prints 0 or 1, and po [0x123456 exerciseAmbiguityInLayout] which makes the auto layout manager pick a different solution for an ambiguous layout.
• The minimum set of constraints required to compute a layout without conflicts or ambiguities is the one that is sufficient for placing and sizing each view. The auto layout manager needs enough information to know where to place the view (x, y), and to size the view (width, height).
• Views that have an intrinsicContentSize provide themselves some information about their sizing (width and height, or either). For instance, for a view that has an intrinsic content height such as UILabel or UIButton objects, you do not need to provide a constraint on its height. How that height is used depends on the view’s contentHuggingPriority and contentCompressionResistancePriority. If you set a constraint on the height of that view, the priorities (of the intrinsic content size and the additional constraints) will determine which constraints are used.
• If you add a view but you do not see it, it’s most likely because 0 is a possibly solution for the width and/or the height of that view. It helps when debugging to give those view a minimum size.
• Priorities are mighty important. They are necessary to express things such as: the view should fill in its superview, but if its content is bigger than the superviews size, it should get bigger.
• Changing priorities might be the solution to solving ambiguous constraints. A constraint is required if its priority is 1000. Anything below is not. Priorities work (I think) relatively to each other. Which means that what actual values you use do not matter as much as their relative difference. Although iOS defines 4 constants, so it might be easier to base your priorities on them: UILayoutPriorityRequired = 1000, UILayoutPriorityDefaultHigh = 750, UILayoutPriorityDefaultLow = 250, UILayoutPriorityFittingSizeLevel = 50.
• Working with constraints in a .xib file is tedious. A simple click or drag will trigger updates on the constraints easily break your constraints. You end up linking constraints to IBOutlets to remove them and add new ones, or modify their constant.
• I used to really like .xib files and thought that they were better to code UIs because unlike UIs created programmatically, .xib files give a good idea of how elements are placed relatively to each other. When using auto layout, .xib files make it a lot harder and inconvenient. The ASCII style of constraint creation provides enough visual information about the placement of UI elements. For instance: @"V:|-8-[button1]-16-[tableView]", @"|-[view(==200)]".
• I haven’t managed to find a way to set constraints properly on scrolling elements in the interface builder. That’s why if I use IB, I have a content view as a top level object in IB, and I create the scrollview in viewDidLoad.
• Do NOT set frames and sizes on views with auto layout. Remove all setFrame:.

My personal conclusions so far are:

• I can’t believe how long it took me to get a grasp of auto layout. Now that I’m getting the hang of it though, it’s not that hard and I like the flexibility of it. But there is still a lot I have to learn about it (such as animations, or the options: or metrics: parameters of constraintsWithVisualFormat:options:metrics:views:) and I’m not sure whether it’s a good idea to fully adopt it or not – time will tell.
• Visual format for creating constraints is great.
• Animating views through constraints works well enough for simple translations (panning, resizing). For animations triggered by gestures (animating a view on a panning gesture event) or for rotations/3D effects, it makes the animation sluggish and the programming tedious (removing/adding constraints). So I expect I’ll be using a mix of views with no auto layout, and some with.
• Auto layout makes it very easy to size views based on their natural / intrinsic content size.
• .xib files become less useful. Maybe it’s a good thing if we can get rid of .xib files in projects. If you use AppCode, then no need to switch to Xcode to code your UI.
• I hope Apple won’t make drastic changes to Auto layout
• Watching the WWDC videos of the talks about auto layout really helped. It is worth investing the few hours in watching them.

There are a couple more use cases that I will probably add to the test app: for instance, mixing views with auto layouts and views without, and examples of animations of constraints’ constants.
Any suggestion and sample code welcome!

Another mapnik install, another set of issues… I had a brand new OSX Lion install on my imac, but had a few issues with the brew install of Mapnik as I wanted to link it to Cairo.
See the Homebrew Mapnik instruction page

brew install python
Change the .bash_profile to add export PATH=/usr/local/bin:/usr/local/sbin:$PATH.


brew install git
brew install postgresql
brew install icu4c
brew install proj
brew install libtiff
brew install gdal --with-libtiff=/usr/local/lib
brew install cairo


The real problems came with cairomm.

brew install cairomm
which led to those errors

Then the issue was that the symbols from cairo were not found.
"_cairo_xlib_surface_get_xrender_format", referenced from:
Cairo::XlibSurface::get_xrender_format() const in xlib_surface.o
ld: symbol(s) not found for architecture x86_64


One trick I found was to check what was required for the cairo library:
pkg-config --cflags --libs cairo

It complained about missing dependencies. I ended up installing also
brew install fontconfig

I ended up with this new Formula:

and cairomm installed properly.

Then finally
brew install mapnik

For my work on Kemtro, I recently explored how I could extract timetable information available in PDF files. There are still many transport agencies that do not want (or can’t for various reasons) make their data available in a friendly format (GTFS).

I’m not aware of any easy and centralized way of doing that, so I did what I typically do when it comes to ‘liberating’ data: I first scraped the PDF files.
But then it appears that extracting the spreadsheet content from a PDF is not that easy. Here are some of the things I looked at (oh and I only have Macbooks, so some of the options are limited)

• Use a PDF library (pyPdf, pdfminer) and do it myself in Python: :( bleh, PDF is a layout format, and after looking a little at the actual PDF markers in the various PDF files I had, it looked like a very bad idea to just even start.
• Use PDF converters applications to convert to Excel format: the only one that kinda worked was PDF to Spreadsheet Pro. You open the PDF file, and you see the content page by page. You can realign red line markers for columns to tell the application how to segment the table. I found that it almost worked but not well enough. Sometimes columns would not be extracted according to the red markers I had set. Plus, it’s impossible to include that into a more automated process.
  I did not try Acrobat Pro because it’s expensive, and there is no Mac trial application available for download.
• Online converters:
  • I first tried the Adobe ExportPDF web app that lets you export PDF files to either Office Work or Excel. I even signed up for it ($20/year). It works decently for single page PDFs, but it often completely fails for multiple pages PDFs – the first page would be decent, but then all the numbers would be included in a single column thus without any formatting at all. For some table format, it does not extract the data properly (one cell will contain the whole line of times, instead of having one cell for a time). Also, the interface is tedious (you have no choice but to upload the file by hand, and the whole conversion takes a long time.
  • Zamzar: it accepts a whole bunch of file formats, and convert them to different ones (not all conversions are possible obviously).
    It works great!
    I’d have a few suggestions to make it easier to integrate it in an automated processing flow, but it’s fast and most of all it has converted almost all the timetables I’ve sent. It also creates one Sheet per page in the resulting Excel document. Almost like all converters I tried, it converts UTF-8 characters to Ascii most of the time (you lose accented characters in the conversion). I signed up for a paid account, and you can send an email with the attached PDF files to a zamzar email that will convert it to the format you want. The only annoying thing is that you have no easy way to download the result files. The web interface is not super awesome, and you have to click on every file to get them back. The only way to automate the process would be to read the emails Zamzar sends you after a conversion finishes as there is a direct download link in the email. If they integrated with dropbox it’d be awesome.
    After sending a hundred+ files, the conversions started to all fail. I’m hoping I was not the one to take their pdf-xls system down, and it took 12 hours for it to be back. Some of the output files could not be open, even with the Excel ‘Open and Repair’.
• Mechanical Turk: That’s actually what I tried first before the online converters. I had to convert each PDF page of the PDF file to an image first because you can’t upload a PDF file when creating a HIT. The HIT description was: “The image contains timetable information that need to be converted into a spreadsheet (google doc spreadsheet, Excel sheet, or CSV file)”, and the reward was $0.30. I created about 20 HITs, and I’ve had mixed results. Most of the results were ok. I suspect most conversions were made using a software/OCR but I’m not sure. Another disadvantage is that the resulting spreadsheet content will probably be formatted differently in the different files. Also, just making the HIT tasks took time and effort. I first had to find a way to create images from PDF files, which I did using the MAC Automator. I only used the MT web interface which limits a lot what you can do and how fast. I think overall I would maybe consider using MT for files that completely fails to be converted by other means, but it did not seem very practical.

Even after getting a whole bunch of .xls files, the journey is still not over. First of, you have to install Office Excel on your MAC if you want to open most of those files. Numbers won’t work. Also, as I wanted to convert the Excel files to CSV, I could automate the process using Excel VBA. I tried Excel to CSV Converter but it does require Excel anyway (they probably run a VBA macro to convert to CSV) and they do not do multi-sheets .xls files.

I assembled an Excel VBA module that would run through every Excel file in a directory, open and save each sheet as a CSV file. I found most of the code in different places, but all put together it’s there below.
You need to 1- Create an Excel file, 2- Open Menu Tools->Macro->VB Editor, 3- Do Menu Insert -> Module, 4- Paste the code, 5- Change the directory for the input, 6- Run. Some of the VBA functions do not work on a MAC, so it’s limited, but it mostly worked except for corrupted files. It’s also not fully automated (Excel might prompt you with notifications). You should save the file as .xlsm (macro enabled).

As an example, I used a simple PDF file from the Sydney buses, then used Zamzar to convert it to .XLS, and the macro above to extract the .CSV file.

Now obviously, I still have to make the scripts to read the CSV files, parse the data correctly to create the actual route and timetable data…

I updated my portfolio, with more details about my past research projects, as well as my latest project Kemtro – which you can find on the appstore :)

Problem:

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;
createdb: database creation failed: ERROR:  encoding UTF8 does not match locale en_US
DETAIL:  The chosen LC_CTYPE setting requires encoding LATIN1.

1
2
3

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;
createdb: database creation failed: ERROR:  encoding UTF8 does not match locale en_US
DETAIL:  The chosen LC_CTYPE setting requires encoding LATIN1.

Check what locales you have

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
POSIX

1
2
3
4
5
6

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
POSIX

If locales are not installed:

# apt-get install debconf

1

# apt-get install debconf

To add en_US.UTF8, run:

# dpkg-reconfigure locales

1

# dpkg-reconfigure locales

And select the locales you want (en_US.UTF8)

Now:

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
en_US.utf8
POSIX

1
2
3
4
5
6
7

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
en_US.utf8
POSIX

You need to restart postgres:

# /usr/lib/postgresql/8.4/bin/pg_ctl restart -D /var/lib/postgresql/8.4/main

1

# /usr/lib/postgresql/8.4/bin/pg_ctl restart -D /var/lib/postgresql/8.4/main

Now rerun:

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;

1

(postgres)-# create database "newdb" WITH ENCODING = 'UTF8' LC_CTYPE='en_US.utf8' LC_COLLATE='en_US.utf8' TEMPLATE template0;