This guide is my notes on keeping several subversion repositories happy that are currently growing by 1k-2k commits per month.
My current approach is based on Ubuntu 8.04 LTS, Apache 2.2 and Subversion 1.5.
I have considered upgrading to Subversion 1.6 for further speed improvements and features, but so far that hasn’t been needed. Additionally most clients accessing at the server are 1.6 clients where the features are more important.
No drastic measures have been taken about multiple subversion servers with a write master and read slaves, currently all activity is located locally on one machine with a 1TB raid.
These tips are mainly targeted at tweaking Apache.
Keep Alive
Due to way we’re using SVN, HTTP, the client will make lots of requests back to back to pull down a update or initial checkout. Essentially one request per file. To make this as efficient as possible, you can instruct Apache to allow keep-alive transactions, which allow for a single TCP session the client to continue to make requests.
Disable SVNPathAuthz
SVNPathAuthz is enabled by default in mod_dav_svn. For the software to determine if a “user” can see a path, it makes several gets (per folder/file) back to Apache to determine if they are actually visible for that user. If you do not perform per directory access controls (inside a single repository), you can safely turn this off. You will see a noticeable improvement on SVN Log information. In my tests our 15k commit repo went from 4 mins to do a svn log for the entire repo to 17 seconds (in an un-cached scenario with a 1.6 client). I picked up this tip from tty1.
Enable LDAP Cache
We currently back Subversion to Active Directory over LDAP. Instead of each request round tripping to the Active Directory pool, we’ve enabled LDAP caching on the Apache side. By default mod_ldap has some caching enabled, normally 10 minutes with a certain memory size for the cache. We’ve found some small gains by drastically increasing the TTL’s from 10 min to 24 hours, and increasing the cache memory size to be up to 1gig. The idea is that, once a user accesses the site, we no longer have to round trip for the rest of that day’s operation.
As an additional note about Active Directory. Make sure that you have some DNS entry you can refer to that only contains Active Directory servers local to the Subversion server. For awhile had a few requests that would attempt to authenticate users to an Active Directory across the pond.
If you are having issues getting the Samsung Trance SCH-u490 or any other MTP USB device to connect to Windows XP, install the latest Windows Media Player. MTP based device support was added to WinXP by Windows Media Player 10.
If you don’t do this, the phone will come up as a USB mass storage device that is unable to start.
I assume you know how to setup and configure Apache2 and Subversion on Ubuntu already. If not go here.
You should already have apache2 and subversion installed through Ubuntu’s package management.
The following overwrites the existing Subversion install, but saves you the hassle of configuring the module system in Apache.
Warning
I’m still testing this approach before rolling it out for even my own general use, but initial testing appears positive.
# install tools and libraries to compile
sudo apt-get install make apache2-threaded-dev autoconf libtool libdb-dev
# get subversion release source and extract
wget http://subversion.tigris.org/downloads/subversion-1.5.5.tar.gz
tar xzvf subversion-1.5.5.tar.gz
cd subversion-1.5.5
# configure to use Apache apxs
./configure --with-apxs=/usr/bin/apxs2
# build and install
make clean && make && sudo make install
# have the system reload dynamic objects
sudo /sbin/ldconfig -n /usr/lib/apache2/modules
# At this stage you should update your repositories using
# sudo svnadmin upgrade <repos path>
# restart apache
sudo /etc/init.d/apache2 restart
For my work on SPNEGO I’ve found that Safari does not play well with the Negotiate WWW-Authenticate message that needs to passed down to indicate a client can authorize with either NTLM or SPNEGO.
What appears to happen is that the Safari continuously requests the address location until some arbitrary timer is hit on the Safari side where it gives up.
I’m seeing this behavior with
- Windows Safari 3.1.2(525.21)
- Windows Safari 3.2.1(525.27.1)
It doesn’t matter if Negotiate is presented as the only authentication method, or if its presented along side Basic auth. Either way the browser behaves in the same way. I’d expect the client if unable to handle an authentication protocol, would simply display the 401 page provided from the request and stop. I’ve been verifying this behavior using Wireshark.
Haven’t yet found a explicit entry for this issue on Windows, but seen references to issues with NTLM on OSX
So far my only solution is to carefully examine the user-agent, but this solution isn’t elegant. I essentially disable Safari from working correctly if it ever does get fixed.
Update – New approach
So another approach has occurred to me, since I’m appending Tomcat session notes to the user, primarily to track the gss-api connection for Kerberos Auth in case it takes more then one packet to send the Kerberos key, I could also attach an additional value indicating that the user has been presented previously with a WWW-Negotiate and ignored it without sending up a valid response but the same session id. Now this falls on it’s face in some edge conditions, but most edge conditions can be resolved by simply restarting the browser or clearing the cookies for that website on the browser side.
Pass a parameter to the VM
-Djava.security.debug="sunpkcs11"
This property is a comma separated list. For additional debug flags to turn on for the security systems in Java, see here
My first availability issue I ran into on Google AppEngine, it appeared to resolve itself around 10 seconds later, but two sequential requests failed.
Nothing appears to be reported in the AppEngine Status
Update
It looks like there is some anomaly that Google is investigating. Graph Here
Error
207.170.206.103 - - [05/Jan/2009:11:42:08 -0800] "GET / HTTP/1.1" 500 1244 - -
E 01-05 11:42AM 08.253
Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 498, in __call__
handler.get(*groups)
File "/base/data/home/apps/blurry-lunch/6.330217185206397397/iwantlunch.py", line 28, in get
for location in locations:
File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 1324, in __iter__
return self.run()
File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 1657, in run
query_run = self._proto_query.Run(*self._args, **self._kwds)
File "/base/python_lib/versions/1/google/appengine/ext/gql/__init__.py", line 572, in Run
it = bind_results.Run()
File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 860, in Run
return self._Run()
File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 882, in _Run
_ToDatastoreError(err)
File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 1637, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
Timeout