在spark上操作hive时不需要搭建hive环境,只需要从现有的hive集群中hive的conf目录下拷贝 hive-site.xml 到spark的conf目录下即可提交程序运行
出现报错
Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
root cause :是没有给程序指定MySQL驱动包的路径
solution:在提交程序的命令中添加 –jars ${mysql driver path}
spark-submit \ --name HiveContextApp \ --jars /opt/apache-hive-1.2.1-bin/lib/mysql-connector-java-5.1.39.jar \ --class com.rz.spark.HiveContextApp\ --master local[2] \ /root/sql-1.0.jar \