Bacth processes are hard to write-especially when using ageneral language like Java. Batch jobs run every night, making it easy formillions of people to do things like banking, online shopping, querying billinginformation.
Spring Batch is Java Framework that makes it easy to writebatch applications. Batch applications invlove reliably and efficientlyprocessing large volumes of data to and from various data sources (files,databases, and so on). Spring Batch is great at doing
this and provides thenecessary foundation to meet the stringent requirements of batch appliocations.Sir Isaac Newton said, “If I have seen further it is only by standing on theshoulders of giants.” Spring batch builds on the shoulders of one giant inpraticular:
the Spring Framework. Spring is the framework of choice for asignificant segment of the Enterprise Java development market. Spring Batchmakes the Spring programming model – based on simplicity and efficiency –easier to apply for batch applications.
What are batch applications? Batch applications processlarge amounts of data without human intervention. You’d opt to use bacthapplications to compute data for generating monthly financial statements,calculating statistics, and indexing files.
The most common scenario for a batch application isexporting data to files from one system and processing them in another. A batchapplication processes data automatically, so it must be robust and reliablebecause there is no human interaction to recover
from an error. The greater thevolume of data a batch application must process, the longer it takes tocomplete. This means you must also consider performance in your batchapplication because it’s often restricted to execute within a specific timewindow. Every
day, large and complex calculations take place to index billionsof documents, using cutting-edge algorithms like MapReduce. For data exchange,message-based solutions are also popular, having the advantage over batchapplications of being(close to) real time.
The goal of the Spring Batch project is to provide an opensource batch-oritened framework that effectively addresses the most common needsof batch applications.
Spring Batch isn’t a scheduler!
Spring Batch drives batch jobs (we use the terms job, batch,and process interchangeably) but doesn’t provide advanced support to launchthem according to a schedule. Spring Batch leaves this task to dedicatedschedulers like Quartz and cron. A scheduler triggers
the launching of SpringBatch jobs by accessing the Spring Batch runtime ( like Quartz because it’s ajava solution) or by launching a dedicated JVM process( in the case of cron).Sometimes a scheduler launches batch jobs in sequence; first job A, and thenjob
B if A succeeded, or job C if A failed. The scheduler can use the filesgenerated by the jobs or exit codes to orchestrate the sequence. Spring Batchcan aslo orchestrate such sequences itself; Spring Batch jobs are made ofsteps, and you can easily configure
the sequence by using Spring Batch XML.
Should a whole batch fail because of one badly formattedline? Not always. The decision to skip an incorrect line or an incorrect itemis declarative in Spring Batch. It’s all about configuration. Components can trackeverything they do, and the framework provides
them with the execution data onrestart. The components then know ehre they left off and can restart processingat the right place.
Spring Batch processes items in chunks. A job reads andwrites items in small chunks. Chunk processing allows streaming data instead ofloading all the data in memory. By default, chunk processing is single threadedand susally performs well. But some batch
jobs need to execute faster, soSpring Batch provides ways to make chunk processing multi-threaded and todistribute processing on multiple physical nodes.
Partitioning splits a step into substeps, each of whichhandles a specific portion of the data. This implies that you know thestructure of the input data and that you know in advance how to distribute databetween substeps. Distribution can take place by ranges
of primary key valuesfor database records or by directories for files. The substeps can executelocally or remotely, and Spring Batch provides support for multi-threadedsubsteps.
Spring Batch and grid computing
When dealing with large amounts of data—petabytes-a popularsolution to scaling is to divide the enormous amounts of computations intosmaller chunks, compute them in parallel(usually on different nodes), and thengather the results. Some open source frameworks(Haddop,
GridGain, andHazelcast, for example) have appeared in the last few years to deal with theburden of distributing units of work so that developers can focus on developingthe computations themselves. How does Spring Batch compare to thesegrid-computing frameworks?
Spring Batch is a loghtweight solution: all it needsis the Java Runtime installed, whereas grid-computing frameworks need a moreadvanced infrastucture. As an example, Hadoop usually works on top of its owndistributed fle system, HDFS. In terms of features,
Spring Batch provides a lotof support to work with flat files, XML files, and relational database.
分享到:
相关推荐
Jointly developed by SpringSource and Accenture, Spring Batch fills this critical gap by providing a robust and convenient framework for writing batch applications that process large volumes of ...
Additionally, you’ll discover how Spring Batch 4 takes advantage of Java 9, Spring Framework 5, and the new Spring Boot 2 micro-framework. After reading this book, you’ll be able to use Spring Boot ...
Spring Batch是Spring的一个子项目,使用Java语言并基于Spring框架为基础开发,使得已经使用 Spring 框架的开发者或者企业更容易访问和利用企业服务。 Spring Batch 提供了大量可重用的组件,包括了日志、追踪、事务、...
Spring Batch API(Spring Batch 开发文档).CHM。 官网 Spring Batch API,Spring Batch 开发文档
Spring Boot整合Spring Batch的一个小例子,在网上发现这方面的资源比较少,特此将其上传供大家学习。
Spring Batch in Action英文pdf版,最新Spring教科书
spring-batch4.0.0 batch spring-batch集成 spring-batch.jar
spring batch批处理框架和对应的源码资源 rar 可以直接运行的
Spring Batch是一个轻量级的,完全面向Spring的批处理框架,可以应用于企业级大量的数据处理系统。Spring Batch以POJO和大家熟知的Spring框架为基础,使开发者更容易的访问和利用企业级服务。Spring Batch可以提供...
基于Spring Batch的大数据量并行处理 基于Spring Batch的大数据量并行处理
Spring Batch批处理框架Spring Batch批处理框架Spring Batch批处理框架
最近在研究springBoot+springbatch ,按照官网的实例做了一个实例。 最近在研究springBoot+springbatch ,按照官网的实例做了一个实例。
主要给大家介绍了Spring Batch读取txt文件并写入数据库的方法,SpringBatch 是一个轻量级、全面的批处理框架。这里我们用它来实现文件的读取并将读取的结果作处理,处理之后再写入数据库中的功能。需要的朋友可以...
mybatis、springBatch、mysql、quartz、spring、springMVC 部署说明: 本项目为两个数据库,由一个数据库的表向另外一个数据库的表做数据迁移,其中数据库脚本在:/src/main/resources/sql/下面(其中data_rep中的表...
SpringBatch数据库建表语句,存储springBatch批处理过程中需要保存的数据和步骤信息
难得的详细spring batch资料 难得的详细spring batch资料
资源名称:Spring Batch 批处理框架内容简介:《Spring Batch 批处理框架》全面、系统地介绍了批处理框架Spring Batch,通过详尽的实战示例向读者展示了Spring Batch框架对大数据批处理的基本开发能力,并对框架的...
《Spring Batch 批处理框架》全面、系统地介绍了批处理框架Spring Batch,通过详尽的实战示例向读者展示了Spring Batch框架对大数据批处理的基本开发能力,并对框架的架构设计、源码做了特定的剖析;在帮助读者掌握...
使用spring batch需要在数据库建立的几个表——建表语句(BATCH_JOB_INSTANCE、BATCH_JOB_EXECUTION、BATCH_JOB_EXECUTION_CONTEXT、`BATCH_JOB_EXECUTION_PARAMS` 、`BATCH_JOB_EXECUTION_SEQ` 、`BATCH_JOB_SEQ` ...
Spring batch in action,很好的一本书。