Lots of manuals and API references will say stuff like “Returns FALSE on failure” with little to no clarification as to what that means. Though it is usually intuitive, there are frequent cases where a little more attention should be paid. Example: PHP manual page for Memcache::delete.

The method takes two parameters: the key to be deleted and the optional timeout. The second parameter specifies how long Memcache should wait before deleting the key. Like many others, the function “Returns TRUE on success or FALSE on failure.”

The primary use case of this method is obviously just deleting a value stored in memcached. But let’s actually examine the possible outcomes.

• Key is found and successfully deleted. Obvious Success.
• Memcached server cannot be reached. Obvious Failure.
• Key is found, but some sort of network or server glitch prevents it from being deleted. Obvious Failure.
• Key doesn’t exist. How do we classify this? On one hand, the “goal” of calling the method is accomplished – the key is not in Memcache. However, the method itself didn’t technically do what it was supposed to do. Its purpose is to delete a key, and it didn’t. Furthermore, it appears that the developer was mistaken as to the state of the cache. Though in a lot of cases that’s fine, what if the developer is privately counting on the key actually being there (this will be more fleshed out in the second usecase). Matters are further complicated by things like memcachedb – a persistent storage backend that uses the memcache protocol. Here, a missing key could present a serious problem, and the developer should definitely know about it, granted this could just be another argument against putting persistent storage behind a protocol meant for the opposite. One way or another, it’s not clear from the documentation what the method would return in this event.

Things get a little more complicated once the second parameter is invoked. The timeout parameter allows the developer to delay the deletion of the key. The first three bullets above roughly apply the same way with minor obvious adjustments (i.e, in the third bullet replace “being deleted” with “having its ttl adjusted”).

The fourth bullet, however, is even more salient. The setting of a timeout value implies that the developer indeed not only expects the key to be there, but is counting on the key to be there n seconds later.

Naturally, I fully understand that one should rarely COUNT on certain things being in the cache, so the aforementioned concerns will likely be irrelevant in most applications. However, I’m not singling out PHP or Memcache: the same concern applies to plenty of other APIs. I remember wondering what “Failure” meant to the YUI Get utility (I thought it was just a non-200 HTTP code, but directing it to a non-existent URL didn’t seem to trigger it, so it’s unclear), and there are plenty of other cases.

I’m not a fan of returning error codes and having to use huge switch statements to determine what to do in the event of every failure in the application, nor do I advocate throwing finegrained exceptions left and right (the two are nearly identical in my mind). However, I do believe that more care should be taken to document what constitutes a failure and, for functions that don’t have a clearly defined return value, success. Perhaps @failure and @success tags can be added to the docblock spec to facilitate such documentation. For every place in a method where false is returned to signify failure, a @failure block would be added, and likewise for success.

PS: if anybody happens to know the correct answer to my Memcache::delete question, let me know!

array_keys is smarter than it looks

The short description for this function in the PHP docs says “array_keys — Return all the keys of an array.” Unfortunately, that is as far as a lot of people read. However, there is an incredibly useful feature hidden in the optional parameters: “If the optional search_value is specified, then only the keys for that value are returned.” So apart from just dumbly getting all the keys the array contains, you can actually parse out some really useful stuff.

Use Case:

You have a table that lists a bunch of entities, and you have to give the user the ability to perform n actions where n > 1 and the actions are mutually exclusive. Simply defining a bunch of checkbox arrays won’t work, so you have to use radio buttons. (You could use checkboxes and use javascript to ensure the exclusivity, but then you’d be a jackass).

Radio buttons, as you know, group by name. So you have to have N radio buttons per row, each with a different value, and a name that indicates which entity the input pertains to. If you’re really smart, you could make each radio group a member of an array, where the array’s key indicates the ID of the entity:

<input type="radio" name="action[<?= $entity->id ?>]" value="delete" />

When the form is submitted, you’ll end up with $_POST['action'][12] => ‘delete’, etc.

Now that you’ve got your input in a nice, tidy array, you can start the magic:

$deletions = array_keys($_POST['action'], 'delete');
$approvals = array_keys($_POST['action'], 'approve');

Now you have just the ID’s of the entities that need to be deleted or approved. Unless you have some really shitty logic libraries, that should make yoru life incredibly easy. I obviously skipped stuff like validation for the sake of brevity and readability, but I would also recommend mapping your actions to an indexed array, so that your radio value were actually more like 0, 1, 2, 3, etc so as to minimize the amount of data posted. Some would also say that you shouldn’t use PHP short tags, but I really just don’t care.

I should also mention that in PHP 5 array_keys has a third parameter bool $strict, which causes it to use === comparison instead of == when parsing the array. Full manual here.

array_splice’s 4th optional parameter might be its most useful

I found myself somewhat dumbfounded when a co-worker asked me if there was a standard function in PHP to insert a set of values at a given point inside an indexed array – I couldn’t think of it! My intuition drew me towards array_splice, even though I knew that the default behavior of that function was the opposite. Having just used the aforementioned obscure feature of array_keys, I guessed that array_splice would either have a similar feature or would direct me to the manual page for its compliment function. The former was correct.

The 4th parameter of array_splice is $replacement. It is pretty self explanatory, but I’ll hold your hand like a small child and do an example anyway.

<?php
$alphabet = $alphabet_broken = range('a','z');
//default behavior. We just got rid of b and c
$missing = array_splice($alphabet_broken, 1, 2);
//same offset, 0 length so nothing is removed
//the missing letters in $replacemenent
array_splice($alphabet_broken, 1, 0, $missing);
//if I don't suck at life, the two should be identical
if (array_diff($alphabet, $alphabet_broken)) {
    echo "Something got fucked up!";
}
?>

The full manual entry for array_splice is here.

I’ve also added a human_interval (not precise) function for approximating in the largest possible units. However, I just realized as I was writing this that I forgot to make it round in any way, so it probably comes up with fairly bogus results right now. Fail. Good thing this is just an exercise, and I’ll have another 3 hours on a shuttle to kill on Monday… and on Tuesday… and on Wednesday…

As I said before, I’m also working on a redo of the WordPress theme, as well as another more “grown up” C project.

Also, did not get laid off. Best of luck to all that did. Crazy times we live in.

PS: just saw on my nifty little wordpress toolbar that 2.7 is available. FUCKING ROCK! New design coming with the quickness.

The extension is called “human,” short for “human readable.” The first function, human_interval_precise, simply represents a number of seconds in the largest “precise” time units possible. That is, thousands of seconds are converted to weeks, days, hours, minutes, and seconds as needed. The source is here.

Lessons

I’m obviously still a giant noob, but there are some things that I have picked up along the way (mostly via Crowley) that made things a lot clearer. Hopefully my posting this will help other noobs following in my footsteps.

“Pointers are just numbers”

Yup. Just the number of memory blocks from the start of the segment of memory you’re dealing with. “Pointer math” is really just math: p+5 really is just the position in memory 5 blocks away from the start of p. The size of the blocks is determined by the pointer type, so if you have a char pointer, the blocks are 1 byte.

an array is just a pointer to a set of consecutive memory blocks

The name of the array is just the pointer to the first element. This is particularly important with char arrays. When I do the following in my code, I am just adding output to the end of an already existing “string.”

sprintf(retstr+strlen(retstr), "%dd ", days);

Always remember to initialize your strings

Nothing says “FAIL” like running your PHP function more than once in the same script and seeing it return its result appended to the end of the result from the previous call. A simple

*retstr = 0;

I started writing this entry a few weeks ago and got sidetracked by a bigger and more interesting project/thanksgiving. As a result, I don’t remember what else I was going to write, so I’ll just wrap it up. More C learnings coming soon. It has been a humbling experience to say the least.

Erlang $99,000 (added 8 June 2008) … Java $79,000 Python $78,000 Perl $77,000 Ruby $74,000 COBOL $73,000 (added due to demand)

JavaScript $72,000

ColdFusion $64,000 (8 June 2008)

Delphi $64,000 PHP $64,000 …

Yikes!

Coding wise, I’ve been at a standstill – I had to do a bunch of stuff for work, came up with a mess of ideas that I had to filter and decide what I actually wanted to work on, and then realized I didn’t want to start working on anything else until I finished ked_duffel, or at least got it to be usable. That itself became a bottleneck as I struggled to figure out how to handle variables within files, but I’ve come up with what I think is a relatively elegant, cheap, and maintainable solution, so I’ve started development. I’ll post details as I implement the stuff

The Conference

I have learned quite a bit of stuff. As I work at a place where we don’t have dedicated DBA’s, but are getting to the point where at least minimal optimization and scaling are becoming a necessity, it was great to hear some talks about efficient scaling, caching, and proper monitoring/query analysis and optimization. Of particular interest and use were the memcached talk and the various discussions of stored procedures. Though only useful in the future (it’ll be a while before MySQL 5.1 is available to us), partitioning also seems like something that would help us out a great deal.

All in all, I’m getting my (read: Yahoo’s) money’s worth and having a good time. I have generated lots of ideas on how we can drastically ameliorate a lot of our performance issues and make our database much more maintainable.

One thing that kind of sucks is that a lot of the talks about monitoring/alerting ended up a waste, as they talked about opensource products that we had looked at and ended up developing our own due to scale issues.

This got boring really quickly, especially once I saw that someone had already done something similar, except weak and pointless. If I’m going to upload “code snippets,” why don’t I just run them through enscript while I’m at it and paste the damn output?

Anyway, then I thought “Hey, this flickr plugin I’m using kinda sucks and isn’t very good about letting me pick which size of photo I want to embed.” It’s a global setting. FAIL.

So then I thought “If Crowley’s drunken ramblings about some sort of open standard for embedding media had any actual substance, I could easily improve this.”

They did. He was talking about oEmbed.

I’m gonna cut this story short because I’m fucking burned, but here’s what I posted on Cal Henderson’s website’s discussion:

I’m messing around with oembed on flickr – not seeing a way to specify which size of the image I want to embed. I tried some combinations of changing the URL and maxwidth/maxheight settings in the request, but to no avail. I tried: www.flickr.com/services/oem… www.flickr.com/services/oem… www.flickr.com/services/oem… All three return the same thing: [ the xml output of this request ] Are we not there yet? I was thinking of making a flickr oembed wordpress plugin. It’ll still work, but will be limited to the one size. Is there a way to work this? Would it be possible to set up server side logic @flickr which actually checks the /sizes/foo part of the URL and returns the appropriate URL/size info? I haven’t thought about it long (and it’s late) but that seems to be in line with the standard – would only have to change the url GET var in the request, putting all the logic on the providers side and just spitting out the right image. Cheers. Mike.

As Crowley put it, that is overkill. I know. But I need to learn how to write PHP extensions

I realize that I’ve started more projects in the last month than I have in the last 2 years. I am going to prioritize things a little bit – I want to focus on ked_duffel (the SVN delivery system) to start out, then once that’s functional use it to update mihasya.com’s framework files, then tinker with the framework a little bit, using duffel to test it on mihasya.com.

Once I’m happy with its functionality, I’ll start compiling a pecl skeleton and trying to fill it out, making heavy use of my rollover minutes – to call Crowley with dumb C questions.

Of course, all of this is going to have to wait a little bit – I realized recently that nearly two weeks had passed without me having done any paid work. Someone has to pay for Crowley’s lapdances, so I am going to dedicate a bit more time to Yahoo! stuff in the coming couple of weeks.

I went through several iterations on this code (some of it is in svn, though not all – I had to start the repository over b/c I forgot to add trunk/ branches/ and tags/ ) so here is my approximate thought process.

Step 1: Check the .tpl exists

I should have had this check anyway, but before I did not realize the need – there needed to be an action defined for the view to be rendered anyway, so I focused on that, and that seemed to take care of itself. The following line would just throw an Exception

$reflector = new ReflectionMethod($this, $action);

But now I really do need to actually handle this on its own, since the action might not even exist – WHAT WILLS WAE DO THARN?!

The Wrong Way:

if (ked::$smarty->template_exists($tpl)) { ...

The Right Way:

The right way looks sort of hackish, but bear with me. Smarty is really dumb about how $smarty->display() works. It never returns anything. If it fails to load the template, it prints a warning. Can’t really check to see if it succeeded in any robust way, so this will do:

...
ob_start();
ked::$smarty->display($tpl);
$viewContent = ob_get_contents();
ob_end_clean();
if (!$this->_skipAssign) ked::$smarty->clear_all_assign();
if (strpos($viewContent, "<b>Warning</b>:  Smarty error: unable to read resource:")!==false)
  throw new kedException('The view "'.$this->_view.'" was not found')
...

Step 2: Make _execute Not Choke On Absent Action

The Wrong Way:

This is where I originally threw an exception, but a comment by David Hall made me pause.

...
try {
  $reflector = new ReflectionMethod($this, $action);
} catch (Exception $e) {
  $this->_skipAssign = true; //tell _render to skip smarty->assign related steps for this.
  return false;
}
...

The Right Way:

This caused me to revert to my original intuition for how to do this:

...
if (!method_exists($this, $action)) {
  $this->_skipAssign = true; //tell _render to skip smarty->assign related steps for this.
  return false;
}
...

Step 3 (bonus): Make _render More Efficient

I alluded to this in an earlier code snippet: now that we can know for sure if an action has been executed, we can do this:

...
ob_start();
ked::$smarty->display($tpl);
$viewContent = ob_get_contents();
ob_end_clean();
if (!$this->_skipAssign) ked::$smarty->clear_all_assign();
if (strpos($viewContent, "<b>Warning</b>:  Smarty error: unable to read resource:")!==false)
  throw new kedException('The view "'.$this->_view.'" was not found');
$this->viewContent = $viewContent;
$this->_skipAssign = false; //reset this to make sure subsequent actions don't get tainted
...

What Did We Gain From All This?

Well, I listened to a bunch of really fucking good music on my ipod while doing this, so that’s good enough for me. But seriously, I also ordered like 6 Threadless shirts.

Also, as a side benefit, the framework now supports actionless views without any extra file system operations. It is also wise enough not to bother assigning template variables if the template is static. You also don’t have to define anything special in the controller (in fact, nothing at all is needed there) or in the template to accomplish it – just put the static .tpl file in the right folder under inc/tpl/views.

I also used the word “tainted” in the comments… which is pretty close to “taint”… which is the area between the ass and the balls. That’s a wrap on the day.

So as I was working on mihasya.com, I was simultaneously tinkering with ked_core – a natural thing to happen, since mihasya.com is a test for how ked_core actually does when used in practice (it kicks ass, fyi). I got the idea to permit actionless views (more on how, once again, I did that better than you later), so I got all excited and immediately hacked up the code. So now I had this code that was directly applicable to mihasya.com (a lot of the inner pages don’t do much, so don’t need a controller function defined) in the ked_core svn, but not on the site.

The problem is: now that ked just ships as a big hunk of folders with the internal files and the site’s own files mixed in, the separation gets tricky. Also, my blog/ folder (the WordPress install) had to go under mihasya.ked/www/blog, as that was what my mihasya.com folder was symlinked to… Long story short, I straight up deleted mihasya.ked. That was fine, I thought, because I still had all that code in svn…. OOOOPS. Thank goodness for database storage.

Where This Leaves an Ambitious Mother F@#$%#

Well, this is a problem I gots ta solve. It’s plagued me for years, as applications come shipped in huge tarballs that just extract like they’re kings of the world. If you want to selectively update something, you have to cp files one at a time. Fuck that.

How do I elegantly, and from within the framework, permit the developer to upgrade ked files, but not affect his own? This could be solved easily by some sort of package manager like apt or rpm, but we don’t have that luxury. Or rather, I won’t give myself that luxury. This will be solved with bash, php, and svn in the spirit of minimalistic requirements. After all, it has to work on Dreamhost

What I Intend To Do

This is how it’s goin down, homegirls: I am going to write a shell script which will fetch a special file from a predetermined svn location (the repository doesn’t have to be static, as I’m sure ked will have thousands of mirrors as it sweeps the market ) The file will contain a listing of files which are internal to ked. It will probably just look something like this:

...
inc/controllers/internal
inc/core
inc/tpl/layouts/internal
...

One problem I’m already seeing before I even start hacking this up is that www/index.php contains configuration information that I refuse to move into a conf file (extra file read) that might get overwritten in this way, but I can probably figure out a way to handle that without quite developing a full blown package manager variable manipulation deal. Or maybe I’ll just do that. Cuz I can, son!

Stay tuned for more on this and other ked shenanigans.