Passing Arrays and Nested Params to url_for in Rails 13

Posted by Ryan Kinderman Wed, 07 Feb 2007 21:12:00 GMT

A few weeks ago (okay, more than a few weeks ago, it took me a while to write this), I discussed the problems involved with passing nested hash parameters to named routes in Rails. My coding pair and I discovered another bug (still using rev 5522) when passing hash parameters to a named route in Rails, this time when the hash contains arrays. For example, consider the following call to a named route: 

person_url(:name => ['Ryan', 'Kinderman'])
In order for the params hash to get decoded properly on the server, the resulting URL must be encoded to look like this: 
"http://someurl.com/people?name[]=Ryan&name[]=Kinderman"
Unfortunately, it gets encoded to look like this: 
"http://someurl.com/people?name=Ryan%2FKinderman"
For those of you unfamiliar with CGI escaping, the %2F translates into the '/' character. So, you end up with a params hash in the controller where params[:name] == ['Ryan/Kinderman']. How disappointing. To get around this in the past, I've chosen to either split the hash value on '/', or use my own encoding of arrays that Rails can handle, and then simply decode them myself within the controller. In the above example, I could have done something like: 
person_url(:name => {0 => 'Ryan', 1 => 'Kinderman})
Of course, without the patch I described a few weeks ago, this kind of thing would not be possible either, because Rails can't encode nested hash parameters.

What I present here is a detailed explanation of the problem, with instructions at the end on how to install my plugin patch to fix it. My explanation and patch address the issues for both nested and array parameters. There are a number of methods involved in the solution to this problem. It may be useful at this point for you to refer to Jamis Buck's excellent articles on the gory details of Rails route recognition and generation.

The Rails methods involved in processing route parameters are listed below, in their general call sequence:

When you call link_to or url_for, either explicitly or through the named route *_url methods, they roughly follow the call sequence depicted in the following diagram to generate the query string for the resulting URL: The problems start in the call to options_as_params. This method is not recursive, and processing nested parameters is a recursive problem. The next issue with options_as_params is not actually in the method, but in the to_param method that it calls. If you look at the Rails implementation of Array#to_param, you'll see that all it's doing is joining the elements into a '/' separated string. This doesn't get processed back into separate array elements when the request is received by the controller. So, in the case when value is an Array instance during a call to options_as_params, the resulting string is encoded incorrectly.

The other specific issue lies in the Route#build_query_string method. Take a look at the method, and notice the part that looks like: 

if value.class == Array
  key <<  '[]'
else    
  value = [ value ] 
end
The check for the Array class causes a problem when passing an array to url_for as an option parameter when that array comes from the params hash from within a controller action (*whew*, that was a mouthful!). This is because what you thought was an array is actually an instance of ActionController::Routing::PathSegment::Result. To be honest, I don't know why this is happening. I looked at the code and realized that it'd take me longer to figure out than what I wanted to spend at the time. However, if someone could explain it to me, I'd love to hear it. In any case, to solve this particular problem, the conditional needs to be changed from a check for only Array to Array and any subclasses using something like the is_a? method.

So, those are the issues involved in why array and nested hash parameters don't work properly in calls to url_for. Rather than going through my solution, I'm offering it as a Rails plugin with full unit test coverage, and plan to submit it as an actual patch to the Rails team, with the code cleaned up a bit more. Maybe there are reasons why this sort of thing isn't supported, but I can't think what they might be. I'll post updates here if and when I get more information on this. If you have comments or questions on this patch or parts of the code, please let me know.

You can install the Rails plugin by typing the following into your command-line: ruby script/plugin install http://svn.ryankinderman.net/nested_params_patch To see the issues I've discussed first-hand, after installing the plugin, take a look at controller_test.rb.

Addendum: I checked, and as of revision 6141 of Rails, the issues covered by this article are still present, and the plugin still fixes them.

Addendum (2007/04/03): I've just got around to confirming that, as rwd's commented, the bug has been fixed. If you're using revision 6343 or later of Rails, you probably aren't going to need this patch. Yay!

Comments

Leave a comment

  1. Lon 25 days later:

    Great work. I had one small tweak to add. The original Rails code would strip out paramenters that had a nil value.

    One small tweak in your code in the toquerystring returns this behaviour.

    elements += process_pair(CGI.escape(key.to_s), value) unless value.nil?

    instead of:

    elements += process_pair(CGI.escape(key.to_s), value)

    But as I see you seem to need this functionality.

    I would definitely prefer to see the original behaviour preserved to keep urls cleaner.

  2. rwd 28 days later:

    It looks like it was fixed in changeset 6343, in response to ticket 7047.

  3. wfisk about 1 month later:

    Thanks, that was very useful.

    On my index page I wanted to be able to filter the reuslts displayed and decided the sort order of the results.

    On my edit/show page I wanted to have previous/next links which allows me to go to the next item in the list depending on the filter and sort order I chose on the index page.

    I set up my filter as a hash, and your plugin allows me to pass this along as a param from page to page. Problem solved.

    Thanks for all the work that you obviously put into it. It works great.

  4. Levent about 1 month later:

    Is there a simple way to work around the issue of stripping out parameters that had a nil value using your plugin but not on Edge rails?

    A simply patch I could apply locally to the plugin?

  5. Levent about 1 month later:

    sorry... i think the first comment answered my question... doh!?

  6. http://www.frederico-araujo.com about 1 month later:

    Awesome work, it saved my life!!!

    saved me so many hours.

    thanks Ryan...

  7. andrerobot 8 months later:

    Ryan!

    You saved my life!!

    Thanks!

  8. Andy Triggs 8 months later:

    Thanks for this Ryan -- very useful.

    To completely preserve the nil functionality (as mentioned by Lon), you also need to limit recursion in the case of nil values, so Hash#process_pair becomes:

    def process_pair(key, value)
  elements = []
  if value.is_a?(Array)
    value.each do |element|
      elements += process_pair((key.to_s + '[]').to_sym, element)
    end
  elsif value.is_a?(Hash)
    value.flatten(key).each do |fk, fv|
      elements += process_pair(fk, fv) unless fv.nil?
    end
  elsif !value.nil?
    elements << "#{key.to_s}=#{CGI.escape(process_option_param_value(value).to_s)}"
  end
  elements
end
  9. Ryan Kinderman 8 months later:

    Thanks for the feedback guys. I believe this issue has been resolved since rev 6343. I'll update my patch if someone provides a test along with the solution.

  10. Chris Rode about 1 year later:

    Hey Guys

    I seem to be getting a similar error in rails 2.1.2. Is anyone else experiencing this/know of a fix?

  11. still buggy over 2 years later:

    rails 2.3 also has this bug

  12. mrj over 2 years later:

    Thanks for making this.

    One note: I got the error

      ...in `evaluate_init_rb': undefined method `configuration' for #<String:0x7fdc0242cda0>

    on server start.

    Fixed by changing load to Kernel.load in init.rb.

  13. snpdc over 2 years later:

    mrj Thanks a lot :) i had the same prob

Comments