Short version:
————–
In my sunspot.yml I used a FQDN ( solr.rkbb.co.uk ). Solr was slow
When I used the server IP (10.18.6.224). Solr was fast.
Setting the scene (you can skip this bit):
——————————————-
I’ve been slowing working on some improvements to our business system at work. Whilst most of it currently runs on MS Access and MySQL, I’m slowing working on moving bits into Ruby on Rails. One of the most important things our current system does is store prices and descriptions for over 200,000 products. Searching that database is a crucial task.
Searching in Rails turned out to be very easy. Sunspot had it working very quickly on my development machine. I also had it running on my production server using the sunspot_solr gem which is meant for development only (but mines a small business, so that’s fine). However, when the server was restarted sunspot_solr needed to be manually restarted which was a pain. I thought I should probably get around to setting up a real solr server and point my application to there. So far, so good, simply: copy the config from my rails app to my new Solr service , set the servers hostname in solr.yml, commit, deploy, it worked!
The problem – Solr was terribly slow!
——————————————-
Re-indexing was slow. I could tell something wasn’t right. Neither my rails server or my new solr server were under load.
I created a new product instead (so that would appear in the solr index).
That was slow, but it worked. Displaying search results was also slow.
Check the logs – wow! Yep, Solr is the slow bit
Started GET "/short_codes?utf8=%E2%9C%93&search=test" for 10.18.6.3 at 2014-10-01 14:28:03 +0100
Processing by ShortCodesController#index as HTML
Parameters: {"utf8"=>"✓", "search"=>"test"}
Rendered short_codes/_navigation.html.erb (1.0ms)
Rendered short_codes/index.html.erb within layouts/application (6.7ms)
Rendered layouts/_navigation.html.erb (1.3ms)
Completed 200 OK in 20337ms (Views: 10.3ms | ActiveRecord: 1.7ms | Solr: 20321.1ms)
No way should Solr take 20321ms to respond.
I tried the search on the solr admin interface and the response was instant, so I knew that solr wasn’t the problem. It must be my code (as always!).
As solr replies over http, I tried querying it from my rails server command line. Also slow. So… maybe it’s not my code… then I tried pinging the solr server from my rails server:
ping solr.rkbb.co.uk
it said replies were coming back in less than 1ms .. but then I realised they were taking about 3 or 4 seconds between each report.
I tried pinging another server … same effect…
then I tried pinging my office router… reports every second, just as fast as I’m used to seeing it. But this was the first time I’d used an IP address and not a FQDN
Then I tried pinging my solr server by it’s address … reports every second!
So, maybe all I have to do is configure my application to talk to solr via the server IP instead of FQDN…
I tried…
Started GET "/short_codes?utf8=%E2%9C%93&search=test" for 10.18.6.3 at 2014-10-02 11:51:49 +0100
Processing by ShortCodesController#index as HTML
Parameters: {"utf8"=>"✓", "search"=>"test"}
Rendered short_codes/_navigation.html.erb (0.9ms)
Rendered short_codes/index.html.erb within layouts/application (8.4ms)
Rendered layouts/_navigation.html.erb (0.8ms)
Completed 200 OK in 27ms (Views: 12.2ms | ActiveRecord: 1.1ms | Solr: 8.3ms)
… and I fixed it 🙂
Well, solr is working great. Now I need to figure out what’s wrong with using FQDNs in my network.
Leave a Reply