Doctrine profiler as Joomla plugin

Posted by Siim on June 22nd, 2010

Joomla has a debug plugin that is able to display different kind of information about page execution – load times in page lifecycle, strings localizations and executed SQL statements. But when using Doctrine, you can’t use this Joomla plugin for SQL statements because Doctrine uses it’s own database connection management, different from Joomla’s.

Fortunately Doctrine already comes with built-in profiler support, we only need to integrate it with Joomla plugins framework. Joomla documentation contains a tutorial on how to create a plugin, it contains also a list of events that are supported by the plugin framework. And there are a wiki page that describes the Joomla API execution order.

I needed onAfterDispatch and onAfterRender events so I created plugin of type system. In onAfterDispatch event I attach Doctrine profiler to the database connection and in the onAfterRender event I collect the SQL statements. Profiler logs different kind of events, but I’m only interested in events that show execution.

Here is the source of the plugin ( I have skipped the not so important parts to keep it shorter).

class plgSystemDoctrine_profiler extends JPlugin
{
	private static $NewLineKeywords = '/(FROM|LEFT|INNER|OUTER|WHERE|SET|VALUES|ORDER|GROUP|HAVING|LIMIT|ON|AND)/';

	private $_outputEnabled = false;
	private $_log4phpEnabled = false;
	private $_logEnabled = false;
	private $_logName = 'doctrine.profiler';

	private $_profiler = null;
	private $_formattedEvents = array();

	public function __construct(&$subject, $config)
	{
		parent::__construct($subject, $config);

		// Initialize from params
	}

	private function getProfiler()
	{
		return $this->_profiler;
	}

	public function onAfterDispatch()
	{
		$this->_log4phpEnabled = $this->_log4phpEnabled && class_exists('Logger');

		if (!class_exists('Doctrine_Connection_Profiler')) return;

		$this->_profiler = new Doctrine_Connection_Profiler();

		$connection = Doctrine_Manager::connection();
		$connection->setListener($this->_profiler);
	}

	public function onAfterRender()
	{
		if ($this->getProfiler() == null) return;

		foreach($this->getProfiler() as $event)
		{
			if ($event->getName() != 'execute') continue;

			$this->_formattedEvents[] = $this->formatEvent($event);
		}

		$this->writeOutput();
		$this->writeLog4php();
		$this->writeLog();
	}

	private function writeOutput()
	{
		if (!$this->_outputEnabled) return;

		// write output to page
	}

	private function writeLog4php()
	{
		if (!$this->_log4phpEnabled) return;

		// Write output to log4php
	}

	private function writeLog()
	{
		if (!$this->_logEnabled) return;

		// Write output to Joomla logs
	}

	private function formatEvent(Doctrine_Event $event)
	{
		$sql = $event->getQuery();
		$sql = preg_replace(self::$NewLineKeywords, "\n\t\\0", $sql);

		$formatted = str_repeat('--', 10)."\n";
		$formatted .= $sql."\n";
		$formatted .= "Params: ".join(', ', $event->getParams())."\n";
		$formatted .= 'Elapsed time: '.$event->getElapsedSecs().' seconds'."\n";
		return $formatted;
	}
}

The usage of the onAfterDispatch event depends on your architecture, in some cases it may be too late for you to attach the profiler in that point. Another options is to use similar approach like Joomla is using with its profiler – usage of global variable $_PROFILER. You can define a similar variable for Doctrine’s profiler and attach it to the connection in the same place where you created it. In that case you need to override the getProfiler() method in the plugin class to return that global variable.

One thing that is really missing in Joomla debug plugin, is the support of writing output to a log file. For example in some cases I don’t want to display debug information on the page (although I can hide it with CSS I guess) or I want to monitor it in the background and review later. So I added logging support to my profiler plugin. It has support for log4php and also for Joomla’s own logging system, in addition to standard output to the page. You can configure it under plugin settings. You can download it from here.

Building Joomla component with Doctrine, admin views – Part 3

Posted by Siim on April 6th, 2010

Now it’s time to add some back-end administration functionality to our little component. In general it is mostly same as in front-end: controllers, views, models. Backend lists also include functionality for sorting and paging so I created a list helper for that to decouple it from the rest of the code.

I created two separate controllers for movies and genres, because controllers have more tasks in the back-end and it’s not very good idea to stuff all that together in a single controller. Each controller contains methods for viewing, editing and saving. When updating database, I’ll use transactions to maintain consistency. Doctrine uses application level transactions and it also supports nesting. So it’s one more plus point for using Doctrine over Joomla DB layer. As far as I know, Joomla default DB layer doesn’t support transactions at all. Here is part of the movies controller:

public function viewMovies()
{
	$listHelper = new ListHelper($this, 'title');

	// Doctrine extracts JosMvMovie objects with all genres it has from this query
	$query = Doctrine_Query::create()
			->select('m.*, g.*')
			->from('JosMvMovie m')
			->leftJoin('m.Genres g')
			->addOrderBy($listHelper->getOrderBy().' '.$listHelper->getOrderByDir())
			->offset($listHelper->getRowIndex())
			->limit($listHelper->getPageSize());
	$items = $query->execute();
	$totalCount = Doctrine_Query::create()
				->select('count(m.movie_id) as count')
				->from('JosMvMovie m')
				->execute();

	$view = $this->getView('MoviesList');
	$view->setViewHelper($listHelper);
	$view->setItems($items);
	$view->setItemsTotalCount($totalCount[0]->count);
	$view->display();
}

public function editMovie()
{
	$movie = $this->getItem();
	// Used by the genres selectbox
	$genres = Doctrine_Query::create()->from('JosMvGenre')->addOrderBy('name ASC')->execute();

	$view = $this->getView('MovieDetails');
	$view->setViewHelper(new ViewHelper($this));
	$view->setLayout('form');
	$view->setGenres($genres->getData());
	$view->setItem($movie);
	$view->display();
}

public function saveMovie()
{
	Doctrine_Manager::connection()->beginTransaction();
	try
	{
		$movie = $this->getItem();
		$movie->title = JRequest::getString('title');
		$movie->imdb_link = JRequest::getString('imdb_link');
		$movie->plot = JRequest::getString('plot');
		$this->updateGenres($movie);
		$movie->save();
		Doctrine_Manager::connection()->commit();

		$this->getApplication()->enqueueMessage(JText::_('Saved'));
		$this->redirectAfterSave($movie);
	}
	catch(Exception $ex)
	{
		JError::raiseWarning(0, JText::_('Save failed').': '.$ex->getMessage());
		Doctrine_Manager::connection()->rollback();
	}
}

ListHelper you saw, is basically a wrapper for using applications state with some magic strings. One of it’s methods looks like this:

public function getOrderBy()
{
	return $this->getController()->getApplication()->getUserStateFromRequest(
		$this->getController()->getName().'.list.filter_order', 'filter_order', $this->_defaultOrderBy, 'cmd');
}

I also created a bunch of base classes for controllers and views, because I don’t like repetitive code that Joomla has all over the place. And in general it is a good idea to write your own base classes between your concrete classes and framework base classes when you have more than a few views. It‘s a good place to put component wide logic that is applied to all views. I have also encapsulated usage of Joomla global variables like $mainframe (which is actually of type JApplication) and $option into base classes. I tried to avoid usage of magic strings and global variables by encapsulating them into methods/properties, making the usage more explicit.

Views are as lightweight as they were in the front-end. They contain only setup of the JToolBar and setters for data which are used by the controllers, nothing more.

<form method="post" name="adminForm" id="adminForm">
<div class="col width-100">
<fieldset class="adminform">
	<legend><?php echo JText::_('Details'); ?></legend>
	<table class="admintable">
	<tr>
		<td width="100" align="right" class="key">
			<label for="name"><?php echo JText::_('Name'); ?>:</label>
		</td>
		<td>
			<input class="text_area" type="text" name="name" id="name" size="32" maxlength="250" value="<?php echo $this->item->name; ?>" />
		</td>
	</tr>
	</table>
</fieldset>
</div>
<?php echo $this->createMetaInputs(); ?>
</form>

In my view base class I have a method createMetaInputs() to create hidden fields, common to all views in Joomla. I also created helper method createHeaderLink(..) used by the list views. DRY.

I think it’s pretty much it. Final result looks much more cleaner and simpler than by using ‘”Joomla’s way”. Final component source code can be downloaded here.

Conclusion

Joomla don’t dictate how you should build your component. This is in some way bad (when looking the source code of the most of the components built by the community), but it’s also good. It allows us to use different (and sometimes better) implementation of the DB layer (and why not other layers too). It also provides pretty good base functionality, you just have to use it wisely.

Building Joomla component with Doctrine, adding views – Part 2

Posted by Siim on March 4th, 2010

In my previous post we set up initial structure for the component and generated models. Now it’s time to add some front-end views. I’ll show you how Doctrine can be used in controllers and how to create clean and simple views.

When using Joomla pattern, then most of the work is done in view’s display() method. I don’t like that. I’d like that my controller provides needed data to the view. So we need to create a base controller from which our component controllers inherit to provide an injection point for our views. View is created in controller’s display() method so we need to override that but most of the code remains same, though.

protected abstract function onViewLoaded(MvViewBase $view);

public function display($cachable=false)
{
	$document = JFactory::getDocument();

	$viewType = $document->getType();
	$viewName = ucfirst(JRequest::getCmd('view', 'topic'));
	$viewLayout	= JRequest::getCmd('layout', 'default');

	$view = $this->getView($viewName, $viewType, '', array('base_path' => $this->_basePath));

	// Set the layout
	$view->setLayout($viewLayout);

	// Display the view
	if ($cachable && $viewType != 'feed')
	{
		global $option;
		$cache = JFactory::getCache($option, 'view');
		$cache->get($view, 'display');
	}
	else
	{
		$this->onViewLoaded($view);
		$view->display();
	}
}

public function getModel($name='', $prefix='', $config=array())
{
	return false;
}

So now controller can be very simple – only load data using Doctrine and initialize the view. Views contains also only setters and getters for the data they need.

class Controller extends MvControllerBase
{
	protected function onViewLoaded(MvViewBase $view)
	{
		switch($view->getName())
		{
			case 'movies':
				$this->dataBindMoviesView($view);
				break;

			case 'movie':
				$this->dataBindMovieView($view);
				break;
		}
	}

	private function dataBindMoviesView(ViewMovies $view)
	{
		$query = Doctrine_Query::create()
				->select('m.*')
				->from('JosMvMovie m')
				->orderBy('m.title ASC');

		$movies = $query->execute();
		$view->setMovies($movies->getData());
	}

	private function dataBindMovieView(ViewMovie $view)
	{
		$movieId = JRequest::getInt('id', 0);
		$movie = Doctrine_Core::getTable('JosMvMovie')->find($movieId);

		$view->setMovie($movie);
	}
}

Doctrine (like most of the ORMs) can load related data automatically so no extra work needed for that. In our case, I can just ask genres from the movie, I don’t need to load them separately. Doctrine also allows tuning how and when relations are loaded – are they lazy-loaded or joined directly when executing query, for example. You can read more from here and here.

Because we have many-to-many relation between movies and genres, automatically generated models need some tuning. We have to specify the relation class name so that Doctrine can detect how it can fill the collections. When looking the definition of BaseJosMvMovie class, which contains mappings with the data model, then there is:

public function setUp()
{
	parent::setUp();
	$this->hasMany('JosMvMovieGenre', array(
		 'local' => 'genre_id',
		 'foreign' => 'genre_id'));
}

We need to add refClass attribute there which defines the class for many-to-many relation.

public function setUp()
{
	parent::setUp();
	$this->hasMany('JosMvGenre as Genres', array(
		 'local' => 'movie_id',
		 'foreign' => 'genre_id',
		 'refClass' => 'JosMvMovieGenre'));
}

Same kind of change needs to be done with the BaseJosMvGenre too, if you want access movies from the genre. Now we only need to add some views to display our movies. They are dead simple.

class ViewMovies extends MvViewBase
{
	private $_movies;

	public function setMovies(array $movies)
	{
		$this->_movies = $movies;
	}

	public function getMovies()
	{
		return $this->_movies;
	}
}

And HTML for that.

<table width="100%" border="1">
	<caption>Movies</caption>
<thead>
<tr>
	<th>Title</th>
	<th>IMDB</th>
	<th>Genres</th>
</tr>
</thead>
<tbody>
<?php foreach($this->getMovies() as $movie): ?>
<tr>
	<td title="<?php echo $movie->plot; ?>"><a href="<?php echo $this->getRouter()->createMovieLink($movie); ?>"><?php echo $movie->title; ?></a></td>
	<td><a href="<?php echo $movie->imdb_link; ?>" target="_blank">IMDB</a></td>
	<td><?php echo join(', ', $movie->Genres->getData()); ?></td>
</tr>
<?php endforeach; ?>
</tbody>
</table>

So this is it for now. Hope it gets you started. In my next post in the series I’ll create views for the back-end, there are things a little bit different, but not much. Full source code with both the views can be downloaded here.

Building Joomla component with Doctrine – Part 1

Posted by Siim on March 1st, 2010

I have done plenty of development on Joomla platform. Although generally it’s pretty good CMS platform for PHP but the part of that I don’t like is it’s MVC implementation. The Model implementation feels just so weird to me. Maybe because I haven’t used to that approach it uses, because I’m more NHibernate guy from .NET space… Anyway, I decided to look for ORMs in a PHP environment and found Doctrine. I did a simple test project with that and decided to use it to develop a simple Joomla component and keep things as simple as possible.

In it’s standard implementation of MVC in Joomla, controller injects needed models to the view and view asks data directly from the models. In other cases (when there is no need for a view) controller uses models directly to accomplish some task. I don’t like that. I’d like when there is just a plain view which is filled with the data by the controller and I would loose the concept of model (in terms of Joomla) wholly. Model layer will be replaced with the Doctrine and it will be used only by the controller not views.

So lets start.

Sample component

For a sample component we use simple movies catalog. It contains models for Movie and Genre. Movie has attributes like name, plot and link to IMDB. Administrators can manage movies and genres from the back-end. From front-end users can browse movies and filter them by genre and name. This should me simple enough to start with. The purpose of this post is to demonstrate how to integrate Doctrine into Joomla, not to demonstrate capabilities of Doctrine. This is the data model:

movies_data_model

Firstly we need to install Doctrine under the Joomla libraries folder. You can download the latest stable version (which is 1.2.1) from here. After downloading extract the contents of the lib directory under Joomla libraries/doctrine directory.

Configuring Doctrine and creating models

First we need to configure Doctrine to use Joomla database settings. Interesting thing here is the table name format. Firstly we use the Joomla configured table name prefix. Secondly we add our own table prefix because we don’t want to use all the Joomla tables (at least in this case). Here is sample code.

$componentTablePrefix = 'mv_';
$config = JFactory::getConfig();
$ormManager = Doctrine_Manager::getInstance();
$ormManager->setAttribute(Doctrine_Core::ATTR_TBLNAME_FORMAT, $config->getValue('config.dbprefix').$componentTablePrefix.'%s');
$ormManager->setAttribute(Doctrine_Core::ATTR_AUTOLOAD_TABLE_CLASSES, true);
$ormManager->setAttribute(Doctrine_Core::ATTR_VALIDATE, Doctrine_Core::VALIDATE_ALL);
$ormManager->setAttribute(Doctrine_Core::ATTR_QUOTE_IDENTIFIER, true);
$conn = Doctrine_Manager::connection(
    $config->getValue('config.dbtype').'://'.
    $config->getValue('config.user').':'.
    $config->getValue('config.password').'@'.
    $config->getValue('config.host').'/'.
    $config->getValue('config.db'), 'default');

Then we use Doctrine to generate object model from the database tables for us. Of course you can create (or update) them manually if you want, there’s a pretty good documentation on that. We use this command to do that.

$modelsPath = JPATH_SITE.DS.'administrator'.DS.'components'.DS.'com_movies'.DS.'models'.DS;
Doctrine_Core::generateModelsFromDb($modelsPath, array('default'), array('generateTableClasses' => true));
Doctrine_Core::loadModels($modelsPath);

Unfortunately Doctrine also adds table prefix to the class name. So table name jos_mv_movie is converted to class name JosMvMovie. Generally I would remove that but currently I leave it as it is.

Creating a component structure

Next we create a bootstrapper script for our component to do all the wiring for us. It will include setup for Doctrine and for our component. We use PHP spl_autoload_register function to provide functionality to load all necessary scripts automatically without the need to scatter our scripts with require statements. Because Joomla uses __autoload function, we also need to re-register Joomla autloader with spl_autoload_register. That’s because __autoload can be used only once but we need different autoload functions – for Doctrine, our component and for Joomla itself.

Note: Because of Joomla isn’t written for PHP5 specifically, you need to explicitly set JLoader::load function to static.

We will use the same bootstrapper for front-end and back-end so we need to create separate configuration options for those. Because in front-end we need to load classes that are found from the front-end directories but also classes that are defined in the back-end, whereas database configuration remains same. And when creating some integration tests you may want to connect to different database.

Here is part of the code for bootstrapper.

class MvBootstrapper
{
    private static $configurationMode;
    public static function configure(ConfigurationMode $mode)
    {
        self::$configurationMode = $mode;
        self::configureJoomla();
        self::configureMovies();
        self::configureDoctrine();
    }
    private static function getModelsPath()
    {
        $modelsPath = JPATH_SITE.DS.'administrator'.DS.'components'.DS.'com_movies'.DS.'models'.DS;
        return $modelsPath;
    }
    private static function configureJoomla()
    {
        spl_autoload_register('JLoader::load');
    }
    private static function configureDoctrine()
    {
        JLoader::register('Doctrine_Core', JPATH_SITE.DS.'libraries'.DS.'doctrine'.DS.'Doctrine'.DS.'Core.php');
        JLoader::load('Doctrine');
        spl_autoload_register('Doctrine_Core::autoload');
        $ormManager = Doctrine_Manager::getInstance();
        $ormManager->setAttribute(Doctrine_Core::ATTR_TBLNAME_FORMAT, self::$configurationMode->getTablePrefix().'%s');
        $ormManager->setAttribute(Doctrine_Core::ATTR_AUTOLOAD_TABLE_CLASSES, true);
        $ormManager->setAttribute(Doctrine_Core::ATTR_VALIDATE, Doctrine_Core::VALIDATE_ALL);
        $ormManager->setAttribute(Doctrine_Core::ATTR_QUOTE_IDENTIFIER, true);
        $conn = Doctrine_Manager::connection(
            self::$configurationMode->getDbType().'://'.
            self::$configurationMode->getDbUsername().':'.
            self::$configurationMode->getDbPassword().'@'.
            self::$configurationMode->getDbHost().'/'.
            self::$configurationMode->getDbName(), 'default');
        Doctrine_Core::loadModels(self::getModelsPath());
    }
    private static function configureMovies()
    {
        MvBootstrapper::registerPath('base', self::$configurationMode);
        MvBootstrapper::registerPath('models/generated', ConfigurationMode::BackEnd());
        spl_autoload_register('MvBootstrapper::autoload');
    }
    public static function autoload($className)
    {
        // Autoload logic
    }
    private static function registerPath($path, ConfigurationMode $mode)
    {
        // Path registration logic
    }
}

Now we need to create startup scripts for our component. It’s nothing special, just a regular component starter page. Here is the code for front-end view.

defined('_JEXEC') or die('Restricted access');
require_once (JPATH_COMPONENT_ADMINISTRATOR.DS.'bootstrapper.php');
MvBootstrapper::configure(ConfigurationMode::FrontEnd());
require_once dirname(__FILE__).DS.'controller.php';
$className = 'Controller';
$controller = new $className();
$controller->execute(JRequest::getCmd('task'));
$controller->redirect();

That’s it for now. In my next post I’ll show how to create controllers and views for the front-end. And how to use Doctrine in controllers and to pass data to a view.

You can download current version of the component from here.


Copyright © 2007 Siim Viikman's blog.