I have been fighting a problem with mounting volumes from an NFS Server in AWS for a few days. With more pressing issues at hand, I had to try and google for solutions for an hour or so before bed, and nothing I was doing was having any effect. Curiously, an Ubuntu based machine that was mounting the drive using NFS3 was not having the same problem. Only the Amazon Linux servers that were using NFS4 were having the issue, and were showing all files and directories as being owned by nobody:nobody.
drwxr-xr-x 2 nobody nobody 22 Jan 9 19:58 installervc
drwxr-xr-x 2 nobody nobody 4.0K Jan 9 19:56 avatar
drwxr-xr-x 2 nobody nobody 4.0K Jan 9 19:56 accessories
I had previously insured that the user UID and group GID for the user that would be writing files (in my case "apache" was the same (with the same UID and GID) on the NFS server and the servers mounting the nfs volume.
As it turned out the problem was with the configuration (or lack thereof) of the rpcidmapd service. NFS4 relies on this service to map users between machines. The "idmapd" requires that the domain of both the client and server should match for the UID/GID mapping to work, and in my case it wasn't. Probably many people with proper DNS configuration don't hit this problem, but we did not have a proper DNS setup, as these machines are part of a growing cluster. Compounding the problem I had set the configuration files to have meaningless host names rather than a domain.
You can tweak this setup by editing the: /etc/idmapd.conf file, and find the "Domain" variable:
Domain = yourdomain.com
Set these to be the same for the server and all the clients.
The last problem was that I had to restart the idmapd process, which has an /etc/init.d control script named /etc/rpcidmapd
[root@web1 init.d]# ls -lath | grep rpcid
-rwxr-xr-x 1 root root 2.7K Jul 28 2011 rpcidmapd
Restart the process on both the nfs server and any nfs clients. If the source of your problem is the same as mine, your user & group mapping problems should be solved:
sudo /etc/inti.d/rpcidmapd restart
Finally!!!!
drwxr-xr-x 2 apache apache 22 Jan 9 19:58 installervc
drwxr-xr-x 2 apache apache 4.0K Jan 9 19:56 avatar
drwxr-xr-x 2 apache apache 4.0K Jan 9 19:56 accessories
drwxr-xr-x 2 nobody nobody 22 Jan 9 19:58 installervc
drwxr-xr-x 2 nobody nobody 4.0K Jan 9 19:56 avatar
drwxr-xr-x 2 nobody nobody 4.0K Jan 9 19:56 accessories
I had previously insured that the user UID and group GID for the user that would be writing files (in my case "apache" was the same (with the same UID and GID) on the NFS server and the servers mounting the nfs volume.
As it turned out the problem was with the configuration (or lack thereof) of the rpcidmapd service. NFS4 relies on this service to map users between machines. The "idmapd" requires that the domain of both the client and server should match for the UID/GID mapping to work, and in my case it wasn't. Probably many people with proper DNS configuration don't hit this problem, but we did not have a proper DNS setup, as these machines are part of a growing cluster. Compounding the problem I had set the configuration files to have meaningless host names rather than a domain.
You can tweak this setup by editing the: /etc/idmapd.conf file, and find the "Domain" variable:
Domain = yourdomain.com
Set these to be the same for the server and all the clients.
The last problem was that I had to restart the idmapd process, which has an /etc/init.d control script named /etc/rpcidmapd
[root@web1 init.d]# ls -lath | grep rpcid
-rwxr-xr-x 1 root root 2.7K Jul 28 2011 rpcidmapd
Restart the process on both the nfs server and any nfs clients. If the source of your problem is the same as mine, your user & group mapping problems should be solved:
sudo /etc/inti.d/rpcidmapd restart
Finally!!!!
drwxr-xr-x 2 apache apache 22 Jan 9 19:58 installervc
drwxr-xr-x 2 apache apache 4.0K Jan 9 19:56 avatar
drwxr-xr-x 2 apache apache 4.0K Jan 9 19:56 accessories
One of the distinctive features of Apple computers is a piece of software called "The Dock". The Dock is sort of an always there menu with big icons that spawn your most frequently used programs. Docks also can run small programs or docklets that can do things like display a clock or show you the weather. Docks also can be configured to display running programs that have been minimized in a mini window. You bring a running program to the front, or launch a program by hovering over the dock with your mouse, and clicking on the icon.
I suppose that people who don't know any better might think that this is something you can only get from Apple, but as it turns out there are docks for Linux and Windows. At home I have a Gateway computer running Windows Vista business, and I wanted to add a Dock. After a bit of hunting, I found some recommendations and settled upon RocketDock from Punk Labs. Punk Labs as it turns out is really a couple (he's a programmer, she's a designer) who go by the pseudonyms of PolyVector and Skunkie respectively. The about screen describes RocketDock tongue in cheek as a "peace offering" from the pair of self admitted Apple fans to Windows users everywhere.
The RocketDock website provides links to the various versions and a straightforward online manual. You get most of the features of the Apple Dock program, and it's very easy to work with. Once installed and running, the RocketDock appears at the top of the screen, with a set of default icons. Right click on the RocketDock and you get a menu that lets you adjust Dock settings. There are various themes that will style the look and feel of the RocketDock, and the RocketDock site offers an Extra's link with downloads to styles and docklet programs other people have contributed. While it's just a quibble with this otherwise great program, there's not much in the way of Docklets available, but I found it hard to resist adding on the simple analog style clock.
Continue reading "Free Apple Mac style Dock with Windows Vista and RocketDock"
The serendipity cache blew up on me for this article so I had to split it into 2 parts.
MySQL knows how to work with the Gregorian calender, and I believe it is easier to let MySQL do the work, rather than trying to figure out how to implement the different cases. I chose to implement the "last day of the month" technique. In this case, we'll do this by creating a mysql date based on March 1st of the current year, and subtract one day from it to get to the last day of February.
If you read my article on "Finding next monday" using mysql date functions, you would have seen how DATE_ADD() can be used to help solve a lot of problems, even though the name can be misleading when you are actually using it to subtract. Using it with the following test dates proves that mysql has fully implemented the leap year logic accurately:
Continue reading "Calculate a person's age in a MySQL query -- continued"
DATE_ADD to the rescue
MySQL knows how to work with the Gregorian calender, and I believe it is easier to let MySQL do the work, rather than trying to figure out how to implement the different cases. I chose to implement the "last day of the month" technique. In this case, we'll do this by creating a mysql date based on March 1st of the current year, and subtract one day from it to get to the last day of February.
If you read my article on "Finding next monday" using mysql date functions, you would have seen how DATE_ADD() can be used to help solve a lot of problems, even though the name can be misleading when you are actually using it to subtract. Using it with the following test dates proves that mysql has fully implemented the leap year logic accurately:
Continue reading "Calculate a person's age in a MySQL query -- continued"
Recently a question was posed on the Phpfreaks.com forums, as to the best way for someone to store a user's birth date using the MySQL database. One person suggested using a varchar as they were most familiar with string functions. My answer was to use the MySQL date type.
I provided a quick "advantages of using date" comparison list:
Storing as a DATE
•A MySQL Date requires 3 Bytes of storage
•You can do Date arithmetic inside mysql (search for dates within ranges) and use mysql functions to calculate values directly in a query
•It intrinsically will only store valid dates
•You can format it in numerous ways
Storing as a Varchar
•A string will require minimum 8 bytes, or 10 with separators
•can't do any form of efficient native range queries
•can't reformat it easily in SQL
•will allow completely invalid dates
I thought a great proof of the benefit of this approach was to show that you could have MySQL calculate the person's current age in a query, using their birthday. Certainly with a string, using PHP for example, you would typically query the database to pull out the string and for each row, turn it into a PHP date and do some calculations in your code, and you'd arrive at the same place, but I wanted to show just how capable SQL -- and in this case MySQL can be especially when you use the native data types and some functions.
Continue reading "Calculate a person's age in a MySQL query"
I provided a quick "advantages of using date" comparison list:
Storing as a DATE
•A MySQL Date requires 3 Bytes of storage
•You can do Date arithmetic inside mysql (search for dates within ranges) and use mysql functions to calculate values directly in a query
•It intrinsically will only store valid dates
•You can format it in numerous ways
Storing as a Varchar
•A string will require minimum 8 bytes, or 10 with separators
•can't do any form of efficient native range queries
•can't reformat it easily in SQL
•will allow completely invalid dates
I thought a great proof of the benefit of this approach was to show that you could have MySQL calculate the person's current age in a query, using their birthday. Certainly with a string, using PHP for example, you would typically query the database to pull out the string and for each row, turn it into a PHP date and do some calculations in your code, and you'd arrive at the same place, but I wanted to show just how capable SQL -- and in this case MySQL can be especially when you use the native data types and some functions.
Continue reading "Calculate a person's age in a MySQL query"
Sometimes I see people attempting to use VARCHARS or CHARS to store dates in their MySQL database application. This is really fighting against MySQL, which has a variety of interchangeable date types. Internally MySQL is storing dates as numbers, which allows it to do all sorts of nice arithmetic and comparisons with very little effort on your part. For example, when you use a mysql DATE column to store a date, you don't have to worry about sortation, or comparing that date to another DATE, because MySQL already understands how to do those things internally. A lot of people also don't realize that they can output a DATE column in just about any way they choose using the DATE_FORMAT function. This causes people to shy away from using DATE, DATETIME, TIME, or TIMESTAMP columns, when they really should.
Continue reading "Exploring Mysql CURDATE and NOW. The same but different."
Continue reading "Exploring Mysql CURDATE and NOW. The same but different."
MongoDB currently does not offer the equivalent of a SQL Update statement when you want to change the value of a string to something that is based on the current value. In my case I needed to find spaces and change them to underscores -- something done easily in php with str_replace(), and in MySQL with REPLACE.
If you had a table named foo with a column named "mystr" you could do this sort of update like this:
UPDATE foo SET mystr = REPLACE(mystr, ' ', '_');
MongoDB does have collection.update() that works with a series of "modifier" operations, but none of them are useful for doing modifications to the current value of a document field when that field is a string.
MongoDB does however support javascript, and the javascript replace() function implements a regular expression search and replace. I was able to accomplish the update using the forEach() method to process the resulting documents one at a time. While not as efficient as having mongo do this internally with a specially built update modifier, at least this method only needs to operate on documents that are known to require modifications, and could be run in the mongo interactive client easily.
Again, assuming the collection is named "foo" and the document field is named "mystr"
db.foo.find({ mystr: /[ ]+/ }).forEach( function(u) { u.mystr = u.mystr.replace(/[ ]/g, "_"); db.foo.save(u); } );
An existing patch promises to add a new update() modifier but until such a time as it exists, this approach can be used when you need to update strings in a collection, referencing the existing values in the transformation.
If you had a table named foo with a column named "mystr" you could do this sort of update like this:
UPDATE foo SET mystr = REPLACE(mystr, ' ', '_');
MongoDB does have collection.update() that works with a series of "modifier" operations, but none of them are useful for doing modifications to the current value of a document field when that field is a string.
MongoDB does however support javascript, and the javascript replace() function implements a regular expression search and replace. I was able to accomplish the update using the forEach() method to process the resulting documents one at a time. While not as efficient as having mongo do this internally with a specially built update modifier, at least this method only needs to operate on documents that are known to require modifications, and could be run in the mongo interactive client easily.
Again, assuming the collection is named "foo" and the document field is named "mystr"
db.foo.find({ mystr: /[ ]+/ }).forEach( function(u) { u.mystr = u.mystr.replace(/[ ]/g, "_"); db.foo.save(u); } );
An existing patch promises to add a new update() modifier but until such a time as it exists, this approach can be used when you need to update strings in a collection, referencing the existing values in the transformation.
I'm doing some symfony work and needed to use the Url helper in a form class to return an error message that included a link. I would consider this one of those exceptions where you really need to be able to have markup in a class rather than in a template.
I should probably mention that this is the way that works in version 1.4.3, and that the method that works in actions will not work in a form class. That method is:
sfContext::getInstance()->getConfiguration()->loadHelpers(...);
That method was deprecated and there is also this article to consider.
In this case, the solution was to access the sfApplicationConfiguration object. In order to use the Url helper so you can call link_to() you need to also include the 'Tag' helper.
sfApplicationConfiguration::getActive()->loadHelpers(array('Url', 'Tag'));
// now you can use link_to(...)
Obviously this would allow you to load other helpers in the same way ... i18n for example.
I should probably mention that this is the way that works in version 1.4.3, and that the method that works in actions will not work in a form class. That method is:
sfContext::getInstance()->getConfiguration()->loadHelpers(...);
That method was deprecated and there is also this article to consider.
In this case, the solution was to access the sfApplicationConfiguration object. In order to use the Url helper so you can call link_to() you need to also include the 'Tag' helper.
sfApplicationConfiguration::getActive()->loadHelpers(array('Url', 'Tag'));
// now you can use link_to(...)
Obviously this would allow you to load other helpers in the same way ... i18n for example.
Dave Child, a Brighton UK based Web developer offers a series of free Reference/Cheat sheets that I think most web developers will find of value. He obviously invested a lot of time and effort into them and it shows. He's created free cheat sheets for MySQL, Javascript, PHP, RGB colors and the subject of today's Gizmola.com entry, Apache mod_rewrite. If you've used Apache there's a good chance you've heard of mod_rewrite, although you might not understand what it can do for you. Lately, a lot of PHP scripts are including functionality that will create mod_rewrite rules and add them to a .htaccess file to support "SEO friendly" url's. Serendipity, which runs my blog, and the popular PHP Content Management System Mambo/Joomla are just two examples of packages that can do this for you.
For example, if your script uses a parameter like this: http://www.yoursite.com/home.php?id=5
A mod rewrite rule can allow users to use: http://www.yoursite.com/home/5/
Many search engines seem to penalize sites that use URL parameters, and are more likely to index and value the non-parameter version. Of course, that's not all that mod_rewrite can do. Many people use it for its ability to offer at least a layer of Hotlink protection, or to handle things like forwarding visitors from obsolete pages that might exist in a search engine to the new ones.
Using and understanding the black magic of mod_rewrite requires a basic understanding of Regular Expressions or as they are typically known: RegEx. If you're new to RegEx, think of them as wildcards on steroids. The October Lampsig meeting will present RegEx basics, and if you're in Los Angeles, definately try and attend the presentation, which will be given by Peter Benjamin. Check out Dave Child's succint reference for this powerful module whether or not you plan to attend Pete's presentation.
Continue reading "RegEx and the Mod_rewrite reference sheet"
Allvideos Reloaded (AVReloaded) is a really nice piece of software engineering from Fritz Elfert. Rather than hardcode for all the individual players out there, Fritz created a macro system that allows you describe the parameters for any player, so it can be configured and updated.
Like the Allvideos plugin, AVReloaded comes bundled with the ubiquitous JW Player for Flash. If you are streaming flash movies from your website, one worthwhile plugin is longtail's Google Analytics Pro. The current license price for the plugin is $19 from longtail's site. If you're already using Google Analytics, the plugin feeds "Content" events from the player to Google Analytics which will show up under the "Event Tracking" menu. Ever have questions like: "I wonder what videos people are actually watching on my site" or "for this new video, how many seconds do they watch before they get bored and close it?" If so, then this plugin is for you. The player will generate events that GA groups into 3 categories: "Video Plays", "Percentage Played" and "Seconds Played". Drilling down you get a nice list of the Videos on your site. For example, finding out that people only watch on average 10% of that promotional video for your product is probably a sign that you may need some better material.
Continue reading "How to configure AVReloaded to work with Longtail's JW Player Google Analytics Pro plugin"
Click for screenshot of Google Analytics Event categories, generated by the Longtail GA Pro plugin
Continue reading "How to configure AVReloaded to work with Longtail's JW Player Google Analytics Pro plugin"
I have a small project that involves an implementation of Joomla (a fork of the popular PHP/Mysql based content management system Mambo). I started with Mambo and during the time I was configuring the site, the core developers of Mambo left to create the Joomla project. The reasons for this are discussed on their OpenSourceMatters website.
Suffice it to say, I find an Entity Relationships diagram to be very helpful in understanding the architecture of a database driven application like Joomla. There wasn't one available, so I decided to reverse engineer the database, and created these diagrams, which break the core database down into a few functional areas.
UPDATED At one time my pdf version of this was hosted by Joomla, but in one of the many moves of source code, the links to it were broken. You can get the pdf with all the views here.
User and Security Subsystem
Backend (modularity and extensibility) Subsystem
Content and Presentation Subsystem
Banners
Logging
Suffice it to say, I find an Entity Relationships diagram to be very helpful in understanding the architecture of a database driven application like Joomla. There wasn't one available, so I decided to reverse engineer the database, and created these diagrams, which break the core database down into a few functional areas.
UPDATED At one time my pdf version of this was hosted by Joomla, but in one of the many moves of source code, the links to it were broken. You can get the pdf with all the views here.
User and Security Subsystem
Backend (modularity and extensibility) Subsystem
Content and Presentation Subsystem
Banners
Logging
Joomla K2 component Schema in png format
One thing I noticed about K2, is that its tree traversal code can require a lot of queries to be run, especially if you have a lot of categories. Over on the K2 forums, one K2 user with a large number of subcategories was complaining about the substantial overhead involved in displaying the top level Category for their site. With over 1000 subcategories, the site was being hammered with an equal number of queries.
This is because by default K2 will do a query for every category to find its child categories and items. Since this is a recursive problem there's no inherent MySQL solution that would allow the developers to solve this problem in SQL (like the Oracle Start With ... Connect By syntax) so it's somewhat understandable that the JoomlaWorks developers provided a short circuit mechanism into the category configuration they call "Catalog Mode". The description of this configuration flag, which defaults to "No" is:
If you select 'yes' then only the items directly belonging to this category will be retrieved. If you select 'no' then items from all sub-categories will be retrieved. Setting this option to 'no' is ideal for news/portal/magazine/blog layouts.
This setting simply shuts off the recursion behavior for the category.
I've been helping out Scott McGinnis with a revamp of his web-videos-for-kids and tweens site, Global Broadcast 4 Kids, aka GB4K.com. Scott was a pretty successful actor in the 1980's, who transitioned to directing in the 90's, and like a lot of directors in Hollywood, got his first chance behind the camera courtesy of Roger Corman. After a stint making low budget features, he went on to direct episodes of "Angel" and the TV series "Honey I Shrunk the Kids", and for the last few years, has been producing educational webisodes using Prosumer video equipment, and editing the episodes on a Mac with Final Cut Pro.
The subject matter and production values aren't that much different from what you might expect to find on basic cable networks like The History Channel or Disney Channel. The GB4K "twist" is that all the segments feature kids and teens as the newscasters, reporters and narrators. GB4K has already proven to be a great training ground for a number of its young "stars", as GB4K alumni have gone on to success at the Disney Channel, and in feature films. There's lots of material in the archive, and he creates a number of new episodes each week, from sports and entertainment news, movie premier "red carpet" interviews, how-to videos on dancing and skateboarding, history channel-esq features on subjects like Guitars and Merry-go-rounds, and environmental science and the latest in Green living practices and Green news. The site also includes music videos he directed and produced for Amber Lily and Jadagrace Berry. A great example of what Scott is able to accomplish as a one-man directing, production and post production crew, is the Jadagrace "Express Yourself" video, which in my opinion compares favorably with music videos costing 10-20x as much.
Despite having a substantial library of video content, frequent updates, and a clear cut audience, problems with the website had put GB4K into a holding pattern.
Continue reading "Profile of the GB4k.com launch -- Joomla reborn"