Solr Cache Autowarming

The Solr index is incrementally updated, i.e. changes are always written to new files. Upon a hard commit a new searcher is created which has a reference to the previous index segments and any new index segments.

The old searcher (pointing to the old index segments) continues handling queries while the new searcher is loaded.

Caches are tied to a specific version of the index therefore new caches have to be autowarmed for the new searcher based upon values of the old cache. You have to pay attention that the warmup time for a searcher is shorter than the time between hard commits. The warmup time can be checked in the Solr Admin's "Plugins/Stats"-page in the "CORE"-section.

New document only become available in a search after the commit plus the time the searcher needs to warmup.

In the Solr Admin's "Plugins/Stats"-page in the "CACHE"-section one can see the "hitratio" and the "warmupTime" for a cache. You want to try for a high hit ratio with a low cache size.

SSH Tunnelling

Some excellent blogs about SSH Tunnelling:

• http://chamibuddhika.wordpress.com/2012/03/21/ssh-tunnelling-explained/
• http://technologyordie.com/ssh-tunneling-and-proxying

About URL encoding

To learn about URL encoding you want to read

http://blog.lunatech.com/2009/02/03/what-every-web-developer-must-know-about-url-encoding

and afterwards you might want to use

http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/net/UrlEscapers.html

Remote Debugging of Jenkins "Maven Project" Jobs

To remote debug a Jenkins "Maven Project" job one has to add

 -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE"   

to the "Goals" line of the Maven Build step. Also see http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html

Note: Do not add it to the JVM-Options line of the Maven Build step. If you do then Eclipse will be able to attach the the remote JVM but it will not stop on a breakpoint.

ModSecurity Rule Execution Order and ctl:ruleRemoveById

In ModSecurity rules are executed in the order in which they are "physically" included into Apache's httpd.config file. First all the rules for phase 1, then all the rules for phase 2 and so on.

The documentation for ctl:ruleRemoveById states that "since this action is triggered at run time, it should be specified before the rule which it is disabling"

Before in this case means that the rule containing ctl:ruleRemoveById needs to run before the rule to be removed.

This means that if the rule to be removed  runs in phase 1 then the rule removing this rule needs to be "physically" included before the rule to be removed.

But if the rule to be removed runs in phase 2 then the rule removing this rule can be "physically" included after the rule to be removed as long as it runs in phase 1.

Spring MVC: Setting 'alwaysUseFullPath ' on 'RequestMappingHandlerMapping' when using 'mvc:annotation-driven'

It seems that the recommended way to set 'alwaysUseFullPath ' on 'RequestMappingHandlerMapping' when using <mvc:annotation-driven /> is to use a 'BeanPostProcessor':

 public class MyBeanPostProcessor implements BeanPostProcessor {  
   private static final Logger logger = LoggerFactory.getLogger(MyBeanPostProcessor.class);  

   @Override  
   public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {  
     if (bean instanceof RequestMappingHandlerMapping) {  
       setAlwaysUseFullPath((RequestMappingHandlerMapping) bean, beanName);  
     }  
     return bean;  
   }  

   private void setAlwaysUseFullPath(RequestMappingHandlerMapping requestMappingHandlerMapping, String beanName) {  
     logger.info("Setting 'AlwaysUseFullPath' on 'RequestMappingHandlerMapping'-bean to true. Bean name: {}", beanName);  
     requestMappingHandlerMapping.setAlwaysUseFullPath(true);  
   }  

   @Override  
   public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {  
     return bean;  
   }  
 }  

See: http://docs.spring.io/spring/docs/4.0.7.RELEASE/spring-framework-reference/htmlsingle/#mvc-handlermapping

PostgreSQL Explain

Some info about PostgreSQL Explain - mainly for my own reference:

1) Include information on buffer usage:

 explain ( ANALYZE, BUFFERS ) select ...  

Tells us how many blocks are read from disc / from the postgres cache.

2) Display shared_buffers size:

 SELECT current_setting('shared_buffers') AS shared_buffers  

Also see: Memory - shared_buffers


3)  Influence the query plans chosen by the query optimizer:

 SET enable_seqscan TO off;  
 EXPLAIN ANALYZE SELECT ...  
 SET enable_seqscan TO on;  

For more options see: Planner Method Configuration

4) Drop Linux page cache and PostgreSQL cache (by restarting PostgreSQL) 

 $ /etc/init.d/postgresql stop  
 $ sync  
 $ echo 3 > /proc/sys/vm/drop_caches  
 $ /etc/init.d/postgresql start  

 5) Update the table's statistics:

 ANALYZE [tablename]  

Stores information in "pg_statistic". Use view "pg_stats" to look at the data

6) Specify an operator class to support like queries on data stored in UTF-8:

 CREATE INDEX ON foo(myColumn text_pattern_ops);

 EXPLAIN SELECT * FROM foo WHERE myColumn LIKE 'abcd%'; 

When you use a encoding other than C, you need to specifiy the operator class (varchar_pattern_ops, text_pattern_ops, etc.) while creating the index.

7) http://explain.depesz.com/


For more information see: Understanding EXPLAIN