2015-01-28

Acquiring JAR files to access Hive server from SQurirrel SQL Client, using Maven's dependency plugin

Hive support JDBC interface, so we can utilize JDBC based SQL client programs such as SQurirrel SQL Client, Eclipse DTD(Data Tools Platform), DBViewer(Eclipse plugin), or even Oracle SQL Developer, to access and execute statements on Hive.

The following wiki page explains that step by step in case of SQurirrel SQL Client

Hive Wiki/HiveServer2 Clients/Integration with SQuirrel SQL Client

Unlike Oracle, MySQL or other most of databases, Hive JDBC driver is not distributed as a self-contained archive, which means lots of other JAR files are necessary to use it.
Above page doesn't explain exactly what files are necessary. And more, explained way to get those JAR files seems not convenient.

The following article kindly explains exact file set in case of Hive 0.10 but doesn't show how to get those files.

Connect to HiveServer2 with a kerberized JDBC client (Squirrel)

Using Maven's dependency plugin, we can download those necessary JAR files in convenient way.

In case of Hive 0.13,x, following JAR files are necessary to use JDBC connectivity.

  • commons-configuration-1.10.jar (Apache Commons Configuration)
  • commons-logging-1.2.jar (Apache Commons Logging)
  • guava-17.0.jar (Google Guava)
  • hadoop-auth-2.4.1.jar
  • hadoop-common-2.4.1.jar
  • hadoop-hdfs-2.4.1.jar
  • hadoop-mapreduce-client-core-2.4.1.jar
  • hive-common-0.13.1.jar
  • hive-exec-0.13.1.jar
  • hive-jdbc-0.13.1.jar
  • hive-metastore-0.13.1.jar
  • hive-service-0.13.1.jar
  • hive-shims-0.13.1.jar
  • libfb303-0.9.1.jar
  • libthrift-0.9.1.jar
  • httpcore-4.3.3.jar (Apache HttpComponents Core)
  • httpclient-4.3.6.jar (Apache HttpComponents Client)
  • log4j-1.2.17.jar (Apache Log4j)
  • slf4j-api-1.7.6.jar (SLF4J API)
  • slf4j-log4j12-1.7.6.jar (SLF4J Log4j Adapter)

The following shell script can be used anywhere just with Maven to get above files.
get goal of dependency plugin can be executed without pom file, so you just need the following script.
The local location to save the downloaded files can be specified by the value of -Ddest option of the mvn command-line in the script.

For Windows, you can use the following batch script.

References

2013-05-13

ToDo

empty

2012-07-19

Documentation Management

Documentation Management

Examples of well-known products

2012-07-18

Implicit type conversion guauantee proper execution plan when using character type value with numeric type column

Implicit type conversion guauantee proper execution plan when using character type value with numeric type column in Oracle

Explanation

Oracle database has lots of implicit type conversion rules and one of them goes like that when comparing a character value with a numeric value, Oracle converts the character data to a numeric value.
This is very important considering execution plan and the performance of query.

For example, with the following query where price column is varchar type.

When executing the above query, Oracle database converts the query like the following and so, there's no problem for the query to use the index on price column. In other words, the above query using character type value with numeric type column doesn't hurt the execution plan of the query and this can be generalized.

Sample

The following is sample table and script showing the above explanation.

Table
Script

DML script to execute query and identify the execution plan used.

Result

The execution of the above script goes like this

For more

2011-11-22

Exception Handling of EJB

EJB 2.1

from Chapter 18. Exception Handling of Enterprise JavaBeans Specification Version 2.1

Exceptions from a Session or Entity Bean’s Business Methods

EJB 3.0

from Chapter 14. Exception Handling of Enterprise JavaBeans Specification Version 3.0

Exceptions from a Session Bean’s Business Interface Methods

2011-10-20

Using SyntaxHighlighter in Redmine

Redmine provides built-in syntax highlighting based on CodeRay (for this, refer here). But it is not so much powerful as leading libraries such as SyntaxHighlighter.

You can use SyntaxHighlighter with Redmine by simply modifying one template file in the Redmine. The file to modify, base.rhtml is a template file specifying the layout of most pages and located at the following position.



The default contents of base.rhtml is like followings.


You can add a few header elements for styles and scripts to install SyntaxHighlighter like the following.

If your Redmine is located at a restricted network and so can reach 'http://alexgorbatchev.com/'. You should download SyntaxHighlighter and deploy it under the public directory of your Redmine installation.

When SyntaxHighlighter is deployed at ${redmine.home}/htdocs/pulbic/vendor/syntaxhighlighter_3.0.83 directory, the custom header elements would be like the following.



You should restart Redmine to apply the change of template. When loading Redmine using Apache HTTP server, just restart Apache HTTP server.

For more about SyntaxHighlighter refer its homepage at http://alexgorbatchev.com/SyntaxHighlighter/

2011-10-13

Guidelines for Multi-moduled Maven Project []

Moved to the page http://3rdstage.wikia.com/wiki/Tips_on_Maven in my wiki as of 23th Jul. 2012.

Guidelines for Multi-moduled Maven Project

Separate parent project from container project.
Define properties in the parent POM file as possible.
Define the following properties explicitly for clearness and control.
  • Encoding related properties whose default value are dependent on platform.
    • project.build.sourceEncoding
    • project.reporting.outputEncoding
  • Properties to skip situationally unnecessary long running goals.
    • skipTests
    • maven.javadoc.skip
    • maven.deploy.skip
    • checkstyle.skip
  • Properties to run maven more stable way.
  • Paths for executable programs external to Maven
    • graphviz.home
Define all plugins in the <pluginManagement> element of parent POM file.
  • Don't specify version to plugin definitions in child POM files.
Define repositories in the parent POM file as possible.
Define plugins to analyze or diagnose the sources or resources of the project as a reporting plugin as possible.
Start values for path with ${project.basedir}, ${project.build.directory}, ${project.build.outputDirectory} or so on
  • When just specifying relative path such as 'target/classes', the result directory could be different if you build the project from the enclosing project in multi-module case.
  • If you specify the path with leading ${project.build.directory}, the resulting directory always would be one that is calculated with the current POM's project.
Use resources in target/classes not in src/main/resources, considering resource filtering.

Sample of Parent POM

This is the real sample of parent POM I am using.

2011-10-10

Frameworks, Libraries, Tools, and Utilities []

Moved to the page http://3rdstage.wikia.com/wiki/Frameworks,_Libraries,_Tools,_and_Utilities in my wiki as of 20th Jul. 2012.

Frameworks

Data Access/OR Mapping
Logging
  • log4jdbc : http://code.google.com/p/log4jdbc/
    a Java JDBC driver that can log SQL and/or JDBC calls (and optionally SQL timing information) for other JDBC drivers using the Simple Logging Facade For Java (SLF4J) logging system.
Templating
  • FreeMarker : http://freemarker.sourceforge.net/
    a "template engine"; a generic tool to generate text output (anything from HTML to autogenerated source code) based on templates.
Testing
  • JUnit : http://junit.org/
    a simple framework to write repeatable tests.
  • TestNG : http://testng.org/
    a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use.
HTML/CSS

Libraries

Java
  • Commons-Collections with Generics : https://github.com/megamattron/collections-generic
    a new version of the popular Jakarta Commons-Collections project that introduces support for Java 1.5 Generics.
  • Guava : http://code.google.com/p/guava-libraries/
    contains several of Google's core libraries that we rely on in our Java-based projects: collections, caching, primitives support, concurrency libraries, common annotations, string processing, I/O, and so forth.
  • Prometheus : http://prometheus.codehaus.org/
    a concurrency library for Java 5 and higher and will be released under the MIT license.
  • Jetlang : http://code.google.com/p/jetlang/
    a complement to the java.util.concurrent package introduced in 1.5 and should be used for message based concurrency similar to event based actors in Scala.
  • JUNG : http://jung.sourceforge.net/
    a software library that provides a common and extendible language for the modeling, analysis, and visualization of data that can be represented as a graph or network.
  • SIGAR : http://sourceforge.net/projects/sigar/
    a cross-platform, cross-language library and command-line tool for accessing operating system and hardware level information in Java, Perl and .NET.
  • javaparser : http://code.google.com/p/javaparser/
    A Java 1.5 Parser with AST generation and visitor support.
Java Graphics
JavaScript
JavaScript Graphics
JSON
Expression Language
  • Java Expression Language (JEXL) : http://commons.apache.org/jexl/
    a library intended to facilitate the implementation of dynamic and scripting features in applications and frameworks written in Java.

Tools

Java Code Analysis
Java Monitoring and Diagnostic
Java Dump Analysis
C/C++ Code Analysis
Web Development
  • Firebug : http://getfirebug.com/
    Edit, debug, and monitor CSS, HTML, and JavaScript live in any web page.
  • JSHint : http://www.jshint.com/
    a static code analysis tool used in software development for checking if JavaScript source code complies with coding rules.
  • JSLint : http://www.jslint.com/
    a JavaScript program that looks for problems in JavaScript programs.
SQL Client/Database Management
System Monitor Tool
Installer
Documentation
Frontend Tools
ETL

Utilities

PC Diagnostic and Recovery
  • Hiren's BootCD : http://www.hiren.info/pages/bootcd
    A bootable software CD containing a number of diagnostic programs such as partitioning agents, system performance benchmarks, disk cloning and imaging tools, data recovery tools, MBR tools, BIOS tools, and many others for fixing various computer problems.
    For download and more information, refer fan & discussion site.
File Synchronization

misc

Graphics Asset

2011-09-16