Big Data File Stage in Datastage 11.x

Hi Mates, As we all know in this era of Hadoop and Big-Data everyone is moving towards working with HDFS. IBM has also introduce Datastage component for Datastage Developer & Designers to access Hadoop Distributed File System via IBM Datastage.

To access HDFS via InfoSphere, we have to first create ishdfs.config file with required classpath details. HDFS Clients .jar and configuration file directories
must be accessible by InfoSphere Server Engine.
If you are using the InfoSphere BigInsights HDFS and using syncbi.sh tool to obtain .jar files.
The ishdfs.config file is created for you automatically from ishdfs.config.biginsights file.
This ishdfs.config file points to the .jar files that are downloaded and unpacked in the $DSHOME/../biginsights directory

Content in File ishdfs.config:
CLASSPATH= $DSHOME/../../ASBNode/eclipse/plugins/com.ibm.iis.client/httpclient-4.2.1.jar:$DSHOME/../../ASBNode/eclipse/plugins/com.ibm.iis.client/httpcore-4.2.1.jar:$DSHOME/../PXEngine/java/biginsights-restfs-1.0.0.jar:$DSHOME/../PXEngine/java/cc-http-api.jar:$DSHOME/../PXEngine/java/cc-http-impl.jar:/opt/IBM/biginsights/IHC/lib/*:/opt/IBM/biginsights/IHC/*:/opt/IBM/biginsights/lib/JSON4J.jar:/opt/IBM/biginsights/hadoop-conf

Location to Save config file ishdfs.config
/opt/IBM/InformationServer/Server/DSEngine

Apart from configuration, other options & operations are almost similar like normal file stage in Datastage where you have to select partitioning method, 
file delimiter and everything else.