Making Magento Commerce Bend to Your Will with Faster Data Imports

Magento provides shop owners with an incredibly sophisticated, feature-rich, and attractive platform with which to do business on-line.

Most web developers within the e-commerce industry are probably familiar with Magento, the ground-breaking e-commerce platform based on PHP and MySQL that has redefined what was previously thought possible with open-source e-commerce solutions. For readers who are unfamiliar with Magento, this platform provides shop owners with an incredibly sophisticated, feature-rich, and attractive platform with which to do business on-line.

Readers who have spent any time at all searching for a suitable e-commerce platform will know that many of the other e-commerce packages available appear to have been “beaten over the head with the ugly stick.” Magento does not fall into that category. Many themes exist that can make this platform appear downright gorgeous. Numerous free and paid plugins, add-ons, and themes are available, and while the company behind this product does push users to explore their paid solutions, a freely available option does exist.

Sounds fantastic, right? Well, rarely in life do all things work to your advantage, and that same truism applies here. While Magento is attractive and feature reach, it is also incredibly slow and bloated under the hood. Without going into too much detail, as that is not the point of this article, let’s just say that Magento’s developers chose to use a very complicated database structure that can cause the platform to come to a screaching hault when dealing with large product inventories.

Many developers have spent countless hours attempting to import large product inventories, attributes, and images into the platform, only to finally give up, swearing to never touch to software again. But never fear! I have seen the light, and I am here to tell you that there is a better way – a solution to Magento’s import woes, and it’s name is “Mage.”

Do a quick Google search on “Magento product import,” and you’re likely to encounter plenty of explanations on how you can use the platform’s .CSV (comma separated values) import and export functionality (found within Magento’s web-based administration console) to accomplish this task. I’ve been down this path, and trust me, it’s a waste of your time. It involves exporting the platform’s sample product data to a .CSV file, analyzing the fields within, and crafting an identically formatted file containing your product data for import.

While that may not sound so bad, those working with larger data sets (think tens of thousands or possibly hundreds of thousands of products) will find this solution to be completely impractical. Crafting your .CSV file for import is the easy part. It’s the actual import process itself that’s painful. While your mileage may vary based on the physical hardware you throw at this problem, I have seen imports of roughly 10,000 products take well over 24 hours. And that’s assuming you don’t run into any problems and have to start over.

Keep searching the web for more practical import methods, and you are likely to encounter how-to’s on using the platform’s SOAP (simple object access protocol) API. While this solution is a step up from the .CSV method of importing product data, as it provides a more intuitive, object-oriented method for importing products, unfortunately it is equally as slow.

So if .CSV export / import functionality and SOAP API calls can’t get the job done when it comes to managing large product inventories on the Magento platform, what will? A little-discussed method that involves using PHP code to directly access the underlying heart of the software (referred to as Mage). The first thing you’ll want to do is create a simple PHP file containing the following code:

require_once("/path/to/your/magento/installation/Mage.php");
umask(0);
Mage::app();
$product = Mage::getModel('catalog/product');
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

Next, you’ll want to construct an array containing all of the product data (title, price, description, etc) that you intend to important. It’s important that each of your products have its own unique identifier (usually a SKU or product ID) for reference. We are now going to loop through each product in our array and check to see if it already exists in our database. If it does not, we will perform a product insert. If it does, we will simply update the product:

foreach ( $products as $p ) {
	
	$product_id = $product->getIdBySku($p['sku']);
	
	if ( $product_id ) {
		
		// The product already exists in our database. Update the product.
		
		$product->load($product_id);

		$product->setUpdatedAt(strtotime('now'));
		
		/*
		At this point, you can define any product settings just as would
		when inserting the product, as seen below.
		*/

		$product->save();
		
	} else {
		
		// The product does not already exist in our database. Insert the product.
		
		$product->setCreatedAt(strtotime('now'));
		$product->setSku($p['sku']);
		$product->setName($p['name']);
		$product->setDescription($p['description']);
		$product->setShortDescription($p['short_description']);
		$product->setPrice($p['price']);
		$product->setTypeId('simple');
		
		/*
		This number may be different for you. You can look this up in your administration panel.
		You are looking for the number of the "default attribute set."
		*/		
		$product->setAttributeSetId('4');

		if ( $cat_id != '' ) {
			/*
			Again, you will need to look this up within your administration panel. Each product category you create has a unique ID,
			and you will want to know what category you want to insert each product into.
			*/
			$product->setCategoryIds($cat_id);
		}

		$product->setWeight($p['weight']);
		$product->setTaxClassId('2');
		$product->setVisibility('4');
		$product->setStatus('1');
		$product->setCost($p['cost']);
		$product->setKeywords($p['keywords']);
		$product->setMetaDescription($p['meta_description']);
		$product->setMetaKeywords($p['keywords']);
		
		// Assign product to the default website.
		$product->setWebsiteIds(array(Mage::app()->getStore(true)->getWebsite()->getId()));
		
		$stockData = $product->getStockData();
		$stockData['qty'] = $p['available_quantity'];
		$stockData['min_sale_qty'] = 1;
		$stockData['is_in_stock'] = 1;
		$stockData['manage_stock'] = 0;
		$stockData['use_config_manage_stock'] = 1;
		$product->setStockData($stockData);
		
		$product->setMediaGallery(
			array (
				'images' => array (
					array (
		                'url'       => "http://www.mightypromos.com/media/catalog/product/your-product-image.jpg",
		                'file'      => "/your-product-image.jpg",
		                'label'     => $p['name'],
		                'position'  => 1,
		                'disabled'  => 0,
		                'removed'   => 0
		            ),
		        ),
	    	)
	    );
	    $product->setThumbnail("your-product-image.jpg");
	    $product->setSmallImage("your-product-image.jpg");
	    $product->setImage("your-product-image.jpg");
		$product->setCanSaveCustomOptions(true);
		$product->setProductOptions($options);
		$product->save();
		
	}
	
}
Sharing is Caring
Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Comment Rules

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i><q cite=""> <strike> <strong>

Kindly use a valid Name and Email address. Spam or unrelated comments will be deleted

If you are a regular commenter, be sure to check our Comment Contest.