Tag Archives: php

php and MongoDB: A ‘find $exists’ Mess of My Own

File this one under “Easy to Overlook … Duh”

Using php with MongoDB to process CSV files, and I want to find all the documents that do not have a particular field. (Inbound, I’m naming the fields a particular way. I’m also using ruby inbound, don’t ask, at least not yet…).

So, I want all documents where a field called UKNOWN_0064 does not exist. So I write:

$m = new Mongo();
$orders_in = $m->$mongo_db->$mongo_coll->find(array("UKNOWN_0064"=>array('$exists'=>'false')));

But I get everything that does have UKNOWN_0064 and none that don’t!

Anyway, you guessed it … the problem is

'$exists'=>'false'

Really needs to be

'$exists'=>false

And so the correct code is:

$m = new Mongo();
$orders_in = $m->$mongo_db->$mongo_coll->find(array("UKNOWN_0064"=>array('$exists'=>false)));

And here I had been feeling so smart to remember to use single tic to pass $exists - and yet blinded by my own ego to miss not enclosing the boolean false.

On a side note, if anyone knows a nicer way to play with MongoDB than all this array business, let me know. It’s clunky to me.

Convert csv to xls (excel) in php

Luckily for me, Ingatius Teo has done all of the heavy lifting, I just had to add a little code around it. Here’s the excel.php class: http://www.phpclasses.org/browse/package/1919.html

When faced with legacy systems, it’s nearly inevitable you’ll come across csv files. In fact, when you’re moving data from anywhere to anywhere, you’re likely to run into csv files.

The challenge I faced here was providing a list of new incoming customers on a daily basis, so that their data could be reviewed in a simpler manner – a spreadsheet to check that fields such as customer classification and email address were being filled in appropriately.

The code shown here is really simple. You include the excel.php class; by including that file you register a stream wrapper for protocol xlsfile – that’s the last thing the excel.php does.

After that, it’s all you. In this working example below, I am running the php program from a command line. Argument 1 is the input file, a tab separated csv file, and argument 2 is the xls file to be written. The code expects the first row to be column headings.

First, we read the first row to set the column headers. Then we reach each line and form an associative array that will be an element of the Big Array ($xlsArray) that contains the data that will become the xls file.

Finally, we write the Big Array, and *poof* we’re done!

/* Get the excel.php class here: http://www.phpclasses.org/browse/package/1919.html */
require_once("../classes/excel.php");
$inputFile=$argv[1];
$xlsFile=$argv[2];
 
if( empty($inputFile) || empty($xlsFile) ) {
    die("Usage: ". basename($argv[0]) . " in.csv out.xls\n" );
}
 
$fh = fopen( $inputFile, "r" );
if( !is_resource($fh) ) {
    die("Error opening $inputFile\n" );
}
 
/* Assuming that first line is column headings */
if( ($columns = fgetcsv($fh, 1024, "\t")) == false ) {
    print( "Error, couldn't get header row\n" );
    exit(-2);
}
$numColumns = count($columns);
 
/* Now read each of the rows, and construct a
    big Array that holds the data to be Excel-ified: */
$xlsArray = array();
$xlsArray[] = $columns;
while( ($rows = fgetcsv($fh, 1024, "\t")) != FALSE ) {
    $rowArray = array();
    for( $i=0; $i<$numColumns;$i++ ) {
        $key = $columns[$i];
        $val = $rows[$i];
        $rowArray["$key"] = $val;
    }
    $xlsArray[] = $rowArray;
    unset($rowArray);
}
fclose($fh);
 
/* Now let the excel class work its magic. excel.php
    has registered a stream wrapper for "xlsfile:/"
    and that's what triggers its 'magic': */
$xlsFile = "xlsfile://".$xlsFile;
$fOut = fopen( $xlsFile, "wb" );
if( !is_resource($fOut) ) {
    die( "Error opening $xlsFile\n" );
}
fwrite($fOut, serialize($xlsArray));
fclose($fOut);
 
exit(0);

Adventures in SOAP and WSDL

Got my first real SOAP challenge: I’ve got to integrate Cybersource’s payment system into our newish RHEL5 servers. The challenge is that our OS is now 64-bit, and we were using their Simple Order 32-bit API in the form of php and perl extensions.

Bzzt. No go. Wisely, Cybersource has decided to go the Web Services direction, so the implementation isn’t all that tricky. Although Cybersource says the SOAP kit is supported by php 5.2.1 and above, 5.1.6 seems to be working just fine for me – at least to their test servers.

Cybersource’s docs say that you should do a php -i | grep soap to see if your php is enabled. Don’t do that. Instead yum install php-soap. Same for php-xml. Install those two and you should be fine.

Another thing I found challenging is figuring out what functions and types are available in the Cybersource interface. There is no documentation – only WSDL for that. So I wrote a quick-n-dirty ditty to dump the functions and types of their WSDL.

Of course 30 minutes later I found out I was going to have to cobble together a SOAP client for a Gift/ Loyalty Card service that we’re implementing.

The long and short of it is, I couldn’t find an easy way to gimme the WSDL details. It’s probably been invented somewhere else, but I needed the practice anyway, so here’s the MXWest gimme-gimme page that will dump WSDL for you. Might even be helpful if you’re writing your own WSDL as it should complain about errors.

Bang it here for Gimme-Gimme a WSDL Dump.

P.S. Yes, I stole Firefox’s background for this little fellow. But it had balloons, and my company is sponsoring the New Jersey Festival of Ballooning this weekend. Tweet me if you’re there @MXWest and stop by the booth!

Where am I? PHP, Pear, Net_GeoIP and Me: Geo-location by IP

Legalese: “This product includes GeoLite data created by MaxMind, available from http://www.maxmind.com/.”

The mission: show our local (in the meastspace sense) visitors a different page from the folks further away. We wanted to show off our store, and attract people to stop in and sign up for photography seminars and classes. (Memo to marketing: perhaps we should measure how effective all this is).  And not annoy our pals in the U.K. who could care less about our coffee bar.

Where to begin? We’re in a state of flux with all of our servers, so we ponied up $250 for an old Dell Poweredge 2500 Server.  We did an Internet installation of Fedora Core 9, because our production and development servers haven’t been updated in a long, long time. Sure, we’d compiled the LAMP basics (from source, for reasons never made clear – perhaps “Because We Could” is the answer), but other than that – well, if it wasn’t broke, we didn’t fix it.

First off, I didn’t know anything about Pear. Luckily I found this article at techrepublic to help me put the pieces together. So Pear is kinda sorta like a cpan tool, but for php instead of Perl. Some re-usable components and so forth.

Install and Configure pear

Anyway, I didn’t have pear installed on my FC9 box. How to install pear? Easy:

yum install php-pear

That was easy. Next, I need to tell php where it can find the pear components I am about to install. Edit /etc/php.ini, and you’ll need to add /usr/share/pear to php’s include_path:

;;;;;;;;;;;;;;;;;;;;;;;;;
; Paths and Directories ;
;;;;;;;;;;;;;;;;;;;;;;;;;

; UNIX: "/path1:/path2"
include_path = ".:/usr/share/pear"

Install Net_GeoIP

pear install Net_GeoIP

My installation complained that there was no “stable version” of Net_GeoIP, and that I should instead download the beta version. Your mileage may vary. I had to do this:

pear install channel://pear.php.net/Net_GeoIP-1.0.0RC1

Of interest: pear’s modules are placed in a structure that is kind of like Java’s com/domain/class structure.  In this case the _ (underscore) becomes a directory separator. That is, to use Net_GeoIP, I need to write this require_once line in my php program:

require_once('Net/GeoIP');

Grab a Location Database

I grabbed the City database from http://www.maxmind.com/app/geolitecity. Please note there are licensing and restrictions of its use. There are both commercial and free versions. Be sure to properly cite MaxMind as per their web site. Note: the file is big. If you are developing using a Windows PC as a middleman, I suggest you wget the database from the Linux command line right into your web server’s filesystem.

I created a subdirectory in my web site’s filesystem called “GeoIP,” into which I copied the GeoLiteCity.dat file.

Program 1: geoTest.php Output

Click here to try out the program.  It will tell you roughly how far you are from this server.

 

Listing 1: geoTest.php

Click here for the program listing. You should be able to just cut and paste the code and with relatively few modifications have a running version of your own.